Pixi’s new iOS app turns text messages into interactive AR experiences

Forget stickers, GIFs, and emoji reactions. Pixi is betting that the next evolution of messaging is interactive augmented reality (AR).

The startup launched its native messaging app on the App Store on Wednesday, allowing users to send AI-powered AR characters through iMessage. Instead of appearing as static media, the characters come to life through the recipient’s iPhone camera, where they can interact with their surroundings, interact with people, and respond in real time.

Although augmented reality isn’t so new and shiny anymore — companies like Snap have been creating AR filters and lenses for years — Pixi believes its approach is different. By combining augmented reality with on-device AI, its characters can understand what’s happening around them and act accordingly. For example, a virtual cat reacts when a real dog passes by. (According to the company, all visual and audio processing operations remain on the device to preserve the user’s privacy.)

Pixi founder Mark Drummond (formerly DreamWorks Animation and formerly Apple) says the app is designed to bring a greater sense of presence and spontaneity to digital conversations. Instead of sending a text message to wish someone a happy birthday, users can send characters that create a shared experience, turning a simple message into something closer to a digital gift or playful interaction.

“The consumer problem we’re solving is thinking about a friend when they’re not around,” he told TechCrunch. “Psychology is sometimes called grit or creative gifts. You share tokens of affection, which are mainly cards and e-cards and gifts. This is your father, or in some cases, your grandfather’s media. We can do better. We can do something that is digitally native, and uses everything we’ve learned about augmented reality on the iPhone.”

Earlier this week, Drummond showed us the app, choosing a cat character, who performed a series of stand-up jokes at his desk. Notably, the cat seemed to respond to Drummond’s facial expressions. For example, the experiment ended when he smiled, demonstrating the character’s ability to perceive emotional cues.

At launch, users will have access to a robot, a cat, and an animated envelope character that can react to their voices and “attack” their friends in a fun way. If they move, the envelope will chase them. There are also games like tic-tac-toe and whack-a-mole.

Pixi plans to expand beyond just a few letters. The goal is to create a marketplace where studios, brands, and independent creators can share their unique personalities for users to choose from. The company envisions this being used at events such as movie premieres or product launches, allowing the characters to generate excitement, such as when M&Ms launch a new flavour.

Drummond also mentioned offering Alice in Wonderland as a character option, as it is open intellectual property. “Our Alice character needs to interact with the things she sees on the desktop in an ‘Alice-consistent’ way,” he noted, showing partners how their creations will interact with technology.

In the future, Pixi hopes to allow users to create their own characters and personalities.

“Part of our plan is to open up generative AI capabilities to us [users]”, so they can get them to say something, like, ‘I want a blue bubble that threatens my friend and growls at them and keeps stalking them on the phone,'” Drummond explained.

To send a character to your friend, download the app on iOS and use iMessage by tapping the plus button in the bottom left corner. No installation is required to receive Pixi message.

Initially, the app will only be available for iPhone 11 models and later, but there are plans to expand to Android devices and messaging platforms like WhatsApp and Instagram in the future.

Also, although the app is free for users, brands will have the option to charge for their characters if they choose.

“We’ll encourage people to do it for free, because then people become your brand ambassadors. You’re making them responsible for using your characters to tell their own stories,” Drummond said.