Not too long ago, tech companies raved about megapixels and hardware capabilities. That’s changing. Out with megapixel talk. While experiencing the new flagship phones from Samsung in San Jose, I realised the impact AI can have in the coming days and months.

There are three new phones from Samsung — Galaxy S24 Ultra, Galaxy S24+ and Galaxy S24. I am not going to get into the branding on the processor or hardware details at the moment because the biggest feature of the new devices is AI.

You may say that Google has promoted a good degree of AI tech with its Pixel phones. It has. But… Google just couldn’t explain how all their AI technology can easily be used.

Samsung, on the other hand, has got the messaging right with AI. In your pocket is a device that you use all the time. It’s as if Samsung is saying: “Here are some AI features on our phones that will make you look sharp in a large group of people.”

AI can remove unwanted shadows and make the photograph appear perfect. Picture: The Telegraph

At the same time, one also needs to remember that Samsung enjoys high brand recall value and brand loyalty in India. When other smartphone brands are slow in showing off their AI capabilities, Samsung is bringing in AI magic on a large scale. We saw some of the new AI-driven features in motion and all we can say is working with the smartphone will not be the same again.

Did we mention that the S24 Ultra comes with a titanium frame making the device lighter but stronger?! Let’s dive into some of the top AI features on the new Samsung phones.

Transcribe Assist

Let’s dive into the Voice Recorder app on the phone. There is a list of recordings and you can choose any. A ‘transcribe’ button will pop up. Next, you need to select the language in which the recording has been made. At once the AI icon comes into play. The time needed for the transcription obviously depends on the length of the recording.

You must be wondering if it is as good as, say, on Otter.AI. I have transcribed a few recordings and the results are spot on.

Let’s take it a step further. I want to translate the transcribed text into Portuguese. Simply select the transcribed text and then the language to which you want to transcribe. I want more: I tried to transcribe the text into French of which I have some knowledge. It works.

There’s more: The voice recording that was transcribed is long. It was like having a TL;DR moment. So, it’s time for Summary. Hit the button and voila. Plus, the summary gets translated into the language of your preference.

Notes Assist

This is another helpful feature that has hints of Microsoft’s Copilot. Here I want to talk about something: The term ‘Copilot’ is slightly boring. Instead of throwing some jargon at you, Samsung has kept it simple. And that matters when it comes to winning over customers.

Open any ‘note’ you may have saved on your phone. When you select the AI button on the bar at the bottom of the screen, there is a list of options to choose from — Auto Format, Summarise, Correct Spelling and Translate. On selecting Summarise, it only takes a few seconds. The same goes for the other options. On selecting Auto Format, headers and bullets get generated. There are a number of formats to choose from. It’s a neat feature and you can generate a report that’s presentable in a matter of minutes.

Visual AI enhancements

Personally, I believe in the sanctity of photographs and I don’t like to play around with it. But AI is here and photography will not be the same again. So, I am looking for ways to figure out if AI has been used in a photograph. Samsung appears to be getting it right.

Using AI, slow-motion can be added to any video and extra frames get generated. Picture: The Telegraph

I selected a photograph of a boy shooting hoops. Hit the edit button and then the AI icon. All you need to do is draw around the object you want to work with. You can use your finger to select it or the S Pen. I traced out the person and then brought him closer to the basket. Hit generate and in a matter of seconds — actually quite a few seconds depending on the complexity of the photograph — the person has moved to a new spot while the background has been tackled by AI. The final image is clean. When you save a copy of the photograph, there is an AI watermark but you can also dive into ‘more info’ and find out if AI has been used.

Chat translation

I have a few Armenian school friends who stay in touch over messages. At times, they tend to message a sentence or two in their mother tongue. It takes me some effort to translate the same. This is where chat translation comes into play. It’s especially helpful if you have a lot of business colleagues spread across geographies.

On choosing the AI icon in the message app, there are three options — Chat Translation, Writing Style, and Spelling and Grammar. Chat translation is what it exactly means. I translated English to French… the entire conversation.

Another interesting feature is translating as you type. So, key in ‘hello, how are you doing’ and see ‘ciao, come stai?’ pop up on the screen. And you can send the translated text.

Circle to Search

There are times when I see my friends enjoying time out in the sun and, at times, I wonder what is the “thing” in the background. Or I see a weird-looking plant or flower and want to know more about it. Hold on to the home key, next circle the photograph you want to learn about, and the results immediately show up. It can also look for celebrities. So next time you won’t mistake an alien for Elon Musk. And it’s not that these photographs need to be shot on the phone. If the picture resides on your phone, you can use Circle to Search on the snap.

Now the interesting bit. At times I meet YouTube influencers who wear shoes that go beyond a journalist’s salary. But there is no harm in finding more information about it. Point at a shoe or a shirt, and use Circle to Search to find out about the object.

Needless to say, some of the results are correct and some are not. When you do Circle to Search in such circumstances, the phone is not saving a photograph to show the results. Why would I like to save the photo of somebody’s shoes?! Once again, neat!

AI improves camera zoom

Samsung has been delivering excellent telephoto snappers on their phones for years. But I wanted something more than 3x or 10x. Most of the time I need a photo taken at 5x. When you look at the updated camera setup, you will notice 0.5, 1x, 3x, 5x and 10x.

All three new phones from Samsung — Galaxy S24 Ultra, Galaxy S24+ and Galaxy S24 — come with a clean design. Picture: The Telegraph

There are moments when I am in a state only cats are found in — too relaxed. I don’t want to get out of the chair to read the QR code at the front of the room or on a display. The level of steadiness and clarity on the camera front has improved.

Does it work with 100x zoom, you know.... space zoom? The steadiness at 100x is better than what it was last year. Hitting 100x, I could manage to scan a QR code that was very far away. And the camera managed to read it. AI has been used to improve zoom at every level.

Slow-mo like you have never before

Often I am found looking for a slow-mo shot while editing videos. And then I realise I didn’t take any. Instant slow-mo uses AI to turn any video into one that has slow-motion tech. You don’t need to shoot in slow-mo mode or have perfect lighting. You can shoot any video and afterwards turn it into a good-looking video. AI analyses motion in any video. What you get next is interpolation, generating extra frames and inserting the same. There are three new frames for every original frame, which means it takes the video down to a quarter speed. You can employ the tech throughout a video that is already on your phone.

Man in the mirror moment

When clicking a photo from behind a glass facade, oftentimes we are stuck with a poor reflection. AI can be used to spot potential problems in any photo. Consider a reflection in a photo. Press the Info button and then Edit suggests options to fix it. One can be something like “do you want to remove reflections”. It takes two-five seconds to fix it. Anyone can fix a photo.

What is far more complicated but executed wonderfully on the new phones is the way shadows are dealt with. Consider a portrait shot and somehow shadows can be seen in parts of the face. AI can adapt and there will be the option to erase shadows. It’s not just about removing the unwanted shadow on the face, AI needs to work on parts of that shadow falling on the wall or on other objects. It’s super-easy and it gets done with on-device computing.

When it comes to generative AI, parts of it get done online. Let’s consider a basketball player who was high in the air, shooting a hoop. Somehow I missed the shot. I can go back in time. Select the person using the S Pen and place him where I want him in the picture. If the edges of the photo need to be generated, AI can do that with generative AI for which a server connection is needed.

Nightography on steroids

Often, I zoom in while taking photos at night. It’s not ideal but that’s how I work. The final photo has noise and shakes. How can AI change that? Samsung already has something called Nightography, which is about improving night shots.

AI is analysing gyroscopic data and that tells you how you are moving. For example, your hand is shaking. AI is able to tell the difference between your movement and the subject’s movements. Then it calculates the differences. By doing so, noise is reduced. Basically, AI is analysing motion live to reduce noise in the dark. So photographs taken at night even when you zoom in, will come out cleaner.

Using Live Translate in real-world

Consider a situation where you need to make a reservation at a restaurant or you can’t find your Uber driver. We tried calling a restaurant. Once the call is made, choose Call Assistant and then Live Translate. Choose the language. Say what you have to say: “Can I reserve a table for eight people on Friday at 8pm?” It takes a few seconds but AI translates what I have said into the language of my choice. The person at the other end hears the translated speech and similarly, his/her dialogues are translated into a language I understand.

AI can be used to translate chats real-time and the processing happens on-device. Picture: The Telegraph

Flawless? I haven’t tried different accents and so on. Plus, the number of languages that are supported is limited at the moment. But the option works well enough to get work done.

(The reporter is in San Jose on an invitation from Samsung, which is taking care of travel and accommodation)