Tapping AI and other software tricks, Microsoft crafts a camera app that it says outdoes Apple’s at taking pictures of people. An Android version is on its way.
Your iPhone has a built-in camera app, but Microsoft hopes to lure you to a new alternative called Pix that’s packed with computing smarts.
Pix combines several advanced technologies: artificial intelligence, computational photography and video stabilization. But it isn’t like Photoshop, with hundreds of options and abilities for you to consider. Instead, it’s designed to improve your photos and videos without you doing any work.
The app, released Wednesday in Apple’s App Store, embodies Microsoft’s new ethos under Chief Executive Satya Nadella. The software maker’s products once served to reinforce the power of its core Windows software. But with Microsoft’s failure to win over the smartphone market, the company now is bringing key programs like Office and Outlook to rival mobile products powered by Apple’s iOS and Google’s Android software.
When you shoot a photo with Pix, it actually takes 10 photos in a half second and presents you with the one it believes is best. If the scene changes significantly, it will present you with a second option and, rarely, three.
The app isn’t just for still photos. It can speed up videos into smooth fast-forward versions with the approach used in Microsoft’s Hyperlapse app for Android. It can also combine the images into distinctive shots called “cinemagraphs” in which parts of the image are still while other parts are in motion.
“Apple has a strong brand in photography,” said Josh Weisberg, principal program manager in Microsoft Research’s Computational Photography Group. But Microsoft believed it could help the iPhone take better photos. An Android version of Pix is on the way. “Now we feel we can do that on any platform,” he said.
The sophistication comes at a cost. It takes up to two seconds to perform all the image processing, and people don’t exactly like waiting for their photos to appear on screen.
How it works
Pix, at this stage, is geared toward taking pictures of people. It tracks faces and tries to set brightness to expose them properly. That’s just the first phase, though. From the 10 photos it takes when you tap the shutter button, it deletes those in which eyes are shut, faces are blurred or people aren’t smiling, Weisberg said.
When you’re shooting photos with Pix, it’s actually taking photos continuously. Microsoft keeps seven photos from before you tapped the button and three from after, because its tests showed that people often missed the moment.
Even though Microsoft discards most of the shots, it first uses them to reduce image noise, or the colored speckles that degrade photos taken in dim or dark conditions. With several frames of the same scene, Pix can zero in on original colors.
In cases where Pix detects a combination of moving and stationary elements, it creates a cinemagraph. For example, it can show people standing still in front of a moving river, or a person’s face unmoving except for blinking. Done right, cinemagraphs can be very eye-catching.
“We intelligently detect when to make them and only make them when it’s interesting,” Weisberg said.
Microsoft isn’t the only company trying to blur the lines between photos and videos. Apple’s newer phones take Live Photos that record a smidgen of video for a more immersive view. Google Motion Stills can stabilize videos and create cinemagraph-like looped animations. And the new Polaroid Swing app captures one second’s worth of your life in what it calls “moving photos.”
Bring on the brains
That intelligence stems from the hot field of machine learning, a type of AI in which computers operate with a digital relative of the human brain called a neural network. Training neural networks to produce good results can take a lot of time. But once a model for making the right choices is built, it can run on an iPhone 5S or later.
AI brings a new facet to a technology called computational photography that’s been steadily gaining in importance. Camera makers have strived to improve their hardware for years, but computational photography processes images after the fact to go beyond the hardware. It combines multiple images into one, corrects lens flaws, gets rid of blurring from a shaky camera and changes the point of focus after the photo is taken.
Deciding when to build a cinemagraph is one situation where Microsoft takes the AI approach. It also uses that approach for understanding a scene to set exposure correctly, to pick the best of the 10 frames and to detect if a person has taken a photo of something stationary like an office whiteboard.
The app is headed for Android phones, though Microsoft won’t say when it’s due out. “Android has more complexity in the variety of camera modules, chipsets and versions of Android,” Weisberg said.
Source: cnet.com
I checked the Microsoft app store and found this is only for iPhones. I think the native camera app already does this with a lens called blink so that’s why it’s not available.
The one thing I love about my Lumia phone over my iPhone is all of the 3rd party camera apps are linked inside of the native app so you only need to open the camera app to add effects. With iPhones you have the native app and then you have to make a separate folder for all of the rest or you will be searching for them
@CraigAF
That’s always been my favourite feature of Windows Phone. The integration of apps as part of the system, not completely separate parts of an app launcher platform.
I’ll likely give this a try, but pretty much everything announced as “new” already exists in the built in iPhone camera app. Computational photography and HDR are already built in… perhaps not used in the sample picture above. Better yet, Apple does this on the ISP, so the results are instantaneous.
Also, I remember when the iPhone 5s came out, it was already doing techniques like taking multiple exposures and automatically selecting the best one every time you press the shutter.
http://www.anandtech.com/show/7329/some-thoughts-about-the-iphone-5s-camera-improvements
Oh… just like Lenses Blink
Love or hate the Lumia line, the cameras were always top in class. It appears they are taking some of the processing and making an ios app. The app osn’t going to make the iPhone camera sensor better but it will get more out of it than the basic camera app.
I like the idea, and it’s probably great for portraits. But the delays associated with the extra processing could be cumbersome when you need to take pictures quickly quickly (kids, sports, etc).
I’ll still give it a shot, and just try to work with the limitations.
@SDarrin
Yeah, all the processing is catered for on Windows Phone with killer camera image processing hardware.
The Nokia purchase was worth it for the PureView platform alone! (If only they could actually sell any phones)
Sounds like it’s worth a try.
I checked the Microsoft app store and found this is only for iPhones. I think the native camera app already does this with a lens called blink so that’s why it’s not available.
The one thing I love about my Lumia phone over my iPhone is all of the 3rd party camera apps are linked inside of the native app so you only need to open the camera app to add effects. With iPhones you have the native app and then you have to make a separate folder for all of the rest or you will be searching for them
@CraigAF
That’s always been my favourite feature of Windows Phone. The integration of apps as part of the system, not completely separate parts of an app launcher platform.
I’ll likely give this a try, but pretty much everything announced as “new” already exists in the built in iPhone camera app. Computational photography and HDR are already built in… perhaps not used in the sample picture above. Better yet, Apple does this on the ISP, so the results are instantaneous.
Also, I remember when the iPhone 5s came out, it was already doing techniques like taking multiple exposures and automatically selecting the best one every time you press the shutter.
http://www.anandtech.com/show/7329/some-thoughts-about-the-iphone-5s-camera-improvements
Okay, I’ve downloaded the app and have taken a range of pictures. The image in this article is deceptive. If you turn on HDR, you get essentially the same image quality on the built in app. Where this app adds an advantage is if you are taking pictures of people. It does a good job of helping you miss the eye blinking, etc. That’s a nice feature that is better than the built in app. However, in terms of overall image quality, I’ve not seen any actual benefit in practice. This app essentially just has HDR on all the time.
@Steve__S I did what you did in an office space with 5 people sitting around the table. I then projected the 6 pictures on a 60 inch flat screen TV, in the same conference room. Of the six pictures the best one chosen by everyone was the PIX, the second best was a PIX chosen by 4 of the 5, the next best was the native Apple, and the next was the PIX.
None of them knew which was taken by the PIX or native app with HDR. All were taken on a iPhone 6. the phone was not mine but I did take the pictures.
Don’t be blinded by the Apple logo try a test like that yourself then comment.
@nutjob
LOL… the moment you go to your 60″ flat screen, you’re already losing quality as compared to your phone. Again, having run my own tests, there was no noticeable difference in quality. The Pix images were different than the standard images, but when I forced the HDR on, the images were essentially the same. You can’t tell me that I’m seeing a quality difference that doesn’t exist. Again, the only benefit I’ve found so far is that the Pix images do a good job of missing people blink.
@Steve__S @nutjob
@Steve__S
Your reply made me laugh. It
showed a lot about you and not in a good light.
1. You do not know how
graphics work if you think (pixel count is all that matters)
“the moment you go to
your 60″ flat screen, you’re already losing quality as compared to your
phone.”
2. Not understanding how
your tests versus independent test are done.
3. You already have your
mind made up when you say things like
“You can’t tell me that
I’m seeing a quality difference that doesn’t exist”
4. Thinking your experiences
are the ones everyone are having is the most tragic and show you to be
extremely self-involved.
I am willing to bet you
aremale between the ages of 16 and 36
who think they know about technology from reading a couple of websites and
taking one or two classes.
Your post just reinforces
how much knowledge you lack.
Frankly, I am moving on and
I hope no one takes your posts seriously.
You want and need it, so you get the last word. Have a nice life.
Oh… just like Lenses Blink
Love or hate the Lumia line, the cameras were always top in class. It appears they are taking some of the processing and making an ios app. The app osn’t going to make the iPhone camera sensor better but it will get more out of it than the basic camera app.
I like the idea, and it’s probably great for portraits. But the delays associated with the extra processing could be cumbersome when you need to take pictures quickly quickly (kids, sports, etc).
I’ll still give it a shot, and just try to work with the limitations.
@SDarrin
Yeah, all the processing is catered for on Windows Phone with killer camera image processing hardware.
The Nokia purchase was worth it for the PureView platform alone! (If only they could actually sell any phones)
@Amusal
LOL… no. For starters, there is nothing magic about PureView. That’s just Nokia’s or now Microsoft’s branding for their post processing tech. Even with that, Nokia had put out phones with nice hardware (large sensors, OIS, etc.), but their post processing was less than desirable. Specifically, Pureview devices have a history of poor handling of white balance, etc. That said, no, this tech is not worth billions… not even close.
Sounds like it’s worth a try.