Samsung’s ISOCELL sensors can record slow motion video at 960fps, reduce the ‘jello-effect,’ capture real-time HDR and have Dual Pixel autofocus. Is this something we will see inside Fujifilm’s X cameras?

While there has always been people on one side or the other of the dispute about the relation “quality versus problems” of Fujifilm’s X-Trans sensor, the truth is that the company is continually improving its own solution and supporting it in ways that, even if the sensor causes some problems at the editing stage, due to its unique architecture, users feel compelled to use the X series of cameras built around the successive versions of the sensor.

In fact, Fujifilm, who has carved its own way in terms of sensors, with the Super CCD series, born in 1999 and used, through its different iterations, until 2010, revealed in 2012 its X-Trans CMOS sensor. The new sensor, representing Fujifilm’s logic transition – albeit a bit late – from CCD to CMOS, continued to promise, essentially, the same as the original Super CCD: more from less!

The Super CCD sensor introduced in 1999 used an array of octagonal photosites in a honeycomb pattern which Fujifilm claimed could gather more light and color information than a conventional CCD sensor with the same number of pixels. In 2012, that’s 13 years later, introducing the X-Trans CMOS, Fujifilm claims the sensor “is capable of delivering resolution that is superior to other APS-C sensors and equal to even some full frame sensors.”

From Super CCD to X-Trans CMOS

Back in 2015 I wrote a long article here at ProVideo Coalition centered on the story of Fujifilm’s sensors, which I’ve followed professionally for a long time. Check “Fujifilm: in search of film’s soul in a digital world” if you want to get a deeper understanding of the path that brought us here, and the recent cooperation between Samsung and Fujifilm.

The recent news that Fujifilm cooperated with Samsung to develop the new ISOCELL Plus technology to enhance light sensitivity and color fidelity in sensors, announced during the Mobile World Congress in Shanghai, held from June 27 to 29 this year, is the footnote to expand on continued rumors started early 2018 suggesting Samsung may produce the fourth generation of X-Trans sensors Fujifilm is expected to use in its next models.

It’s no secret that Sony is the provider of X-Trans sensors for Fujifilm, but apparently the relation has not been the best, and Fujifilm was after an alternative, which may now have a name: Samsung. The news of the recent cooperation, although not immediately associated with larger sensors, do leave enough room for one to speculate. After all, Samsung is no stranger to the universe of sensors, photography and video, which it tried to conquer for some years, before giving it all up, not without leaving some examples of its R&D power on the NX line. Samsung may have exited the photo market, but the company has the knowledge, and Fujifilm could be their next client for sensors.

Building the Samsung ISOCELL brand

Now that we’ve prepared the stage, let’s look at  the new ISOCELL Plus’ technology announced, which allows CMOS image sensors to capture more light, significantly increasing light sensitivity and color fidelity. The technology is presented as ideal for smartphone consumers, who can now expect even more accurate and clearer photos in challenging light environments.

To take high-quality photographs, CMOS image sensors need to hold as much light, or photons, as possible and transmit the right color information to the photodiode. Such requirements were resolved by the introduction of Samsung’s ISOCELL technology in 2013. ISOCELL forms a physical barrier between the neighboring pixels, reducing color crosstalk and expanding the full-well capacity. This enables each pixel to absorb and hold more light than the conventional backside-illuminated (BSI) image sensor design for superior image quality.

Introduced, in 2013, the original ISOCELL technology represents also the brand name that Samsung decided to use for its image sensor lineup. Samsung ISOCELL is underlined by four technological sub-brands—Bright, Fast, Slim and Dual—that respond to specific market demands:

•  ISOCELL Bright
  sensors deliver bright and sharp images with high color fidelity and reduced noise in low light environments
• ISOCELL Fast
  sensors provide fast autofocus onto still or moving objects even when dark
• ISOCELL Slim
  sensors adopt the smallest pixel sizes available in the market at 0.9-1.0um, yet produce high quality images for the slimmest devices
• ISOCELL Dual
  sensors can be mixed and matched in various combinations on consumer devices to bring about features demanded in the latest dual camera trend

“Samsung ISOCELL is a brand that represents the essence of our leading pixel technologies. We expect the ISOCELL brand to help consumers easily acknowledge and confide in camera performance as well as overall quality of the device,” said Ben Hur, Vice President of System LSI marketing at Samsung Electronics. “With our advanced image sensor technologies, Samsung will continue to bring innovation to cameras used in smartphones and other applications.”

Fujifilm’s innovation minimizes light loss

With the introduction of ISOCELL Plus, Samsung pushes pixel isolation technology to a new level through an optimized pixel architecture. In the existing pixel structure, metal grids are formed over the photodiodes to reduce interference between the pixels, which can also lead to some optical loss as metals tend to reflect and/or absorb the incoming light. That’s where Fujifilm comes into the picture. In fact, for ISOCELL Plus, Samsung replaced the metal barrier with an innovative new material developed by Fujifilm, minimizing optical loss and light reflection.

As usual when these things happen, both companies had someone saying something nice about the cooperation. “We value our strategic relationship with Samsung and would like to congratulate on the completion of the ISOCELL Plus development,” said Naoto Yanagihara, corporate vice president of Fujifilm. “This development is a remarkable milestone for us as it marks the first commercialization of our new material. Through continuous cooperation with Samsung, we anticipate to bring more meaningful innovation to mobile cameras.”

“Through close collaboration with Fujifilm, an industry leader in imaging and information technology, we have pushed the boundaries of CMOS image sensor technology even further,” said Ben K. Hur, vice president of System LSI marketing at Samsung Electronics. “The ISOCELL Plus will not only enable the development of ultra-high-resolution sensors with incredibly small pixel dimensions, but also bring performance advancements for sensors with larger pixel designs.”

According to the information now available, the new ISOCELL Plus delivers higher color fidelity along with up to a 15-percent enhancement in light sensitivity. The technology also enables image sensors to equip 0.8-micrometer (µm) and smaller-sized pixels without any loss in performance, making it an optimal solution for developing super-resolution cameras with over 20 megapixels.

From smartphones to other cameras

The three paragraphs above do give us enough suggestions of what the future may be. Fujifilm says that “through continuous cooperation with Samsung, we anticipate to bring more meaningful innovation to mobile cameras”; Samsung adds that “we have pushed the boundaries of CMOS image sensor technology even further” and continues saying that the technology “will not only enable the development of ultra-high-resolution sensors with incredibly small pixel dimensions, but also bring performance advancements for sensors with larger pixel designs”; and finally, we’ve the note that  “the technology also enables image sensors to equip 0.8-micrometer (µm) and smaller-sized pixels without any loss in performance, making it an optimal solution for developing super-resolution cameras with over 20 megapixels”.

We’re talking smartphones here, but one must not forget that the ISOCELL technology is not necessarily exclusive to small devices, and Samsung does mention the technology may bring “performance advancements for sensors with larger pixel designs”. In fact, Samsung had in 2014, according to rumors, a prototype of a new NX camera with an APS-C ISOCELL sensor, a model that never made it to the market, but which continues to feed the rumor mill. After all, many suggest, it makes sense that Samsung uses the technology in more that the small sensors for smartphones. It would mean they could use their R&D for the NX line of cameras and compete with Sony – and Canon, that now, finally, is selling some of its sensors – as a provider of sensors for the industry.

While the recent cooperation with Fujifilm does not necessarily have to mean that the next X-Trans sensor will be provided by Samsung, the simple idea of having larger sensors using ISOCELL technology come to the market would confirm what was said in 2014 about a new NX camera 28MP sensor using ISOCELL. Or, now, ISOCELL Plus. Can a 28MP ISOCELL Plus sensor be the heart of the Fujifilm X-Trans IV sensor?

The new ISOCELL Fast 2L3

There are some interesting aspects of the technology that promise to enhance features important both to videographers and photographers, from super-slow motion to less noise and distortion. Last February  the company introduced its new 3-stack ISOCELL Fast 2L3.  The 1.4-micrometer (μm) 12-megapixel (Mp) image sensor with integrated dynamic random access memory (DRAM) supports super-slow motion, low-light shooting, real-time HDR imaging, while minimizing picture distortion.

“Samsung’s ISOCELL image sensors have made great leaps over the generations, with technologies such as ISOCELL for high color fidelity and Dual Pixel for ultra-fast autofocusing, bringing the smartphone camera ever closer to DSLR-grade photography,” said Ben K. Hur, vice president of System LSI marketing at Samsung Electronics. “With an added DRAM layer, Samsung’s new 3-stack ISOCELL Fast 2L3 will enable users to create more unique and mesmerizing content.”

First, let’s look at conventional sensors, to better understand what the Samsung ISOCELL Fast 2L3 is. Conventional image sensors are constructed with two silicon layers; a pixel array layer that converts light information into an electric signal, and an analog logic layer that processes the electric signal into digital code. The digital code is then sent via MIPI interface to the device’s mobile processor for further image tuning before being saved to the device’s DRAM. While all these steps are done instantaneously to implement features like zero-shutter lag, capturing smooth super-slow-motion video requires image readouts at a much higher rate.

The camera reimagined

The Samsung ISOCELL Fast 2L3 is an advanced solution: a high speed 3-stack CMOS image sensor designed with the company’s two-gigabit (Gb) LPDDR4 DRAM attached below the analog logic layer. With the integration, the image sensor can temporarily store a larger number of frames taken in high speed quickly onto the sensor’s DRAM layer before sending frames out to the mobile processor and then to the device’s DRAM. This not only allows the sensor to capture a full-frame snapshot at 1/120 of a second, but also to record super-slow motion video at up to 960 frames per second (fps) which is 32 times the typical filming speed (30fps).

https://youtu.be/5_-NKRVn7IQ

In addition to super-slow motion video, fast readout can hugely enhance the photography experience. Since the sensor captures an image at very high speeds, it significantly reduces the ‘jello-effect,’ or image distortion, when taking a picture of fast-moving objects, for example helicopter rotor blades. By storing multiple frames in the split of a second, the sensor can support 3-Dimensional Noise Reduction (3DNR) for clearer pictures when shooting in low-light, as well as real time high-dynamic-range (HDR) imaging, and detect even the slightest hint of movement for automatic instant slow-motion recording.

The image sensor is also equipped with Dual Pixel technology, which allows each and every one of the 12-million pixels of the image sensor to employ two photodiodes that respectively work as a phase detection auto-focus (PDAF) agent. With ultra-fast auto-focus in any lighting condition, it will be harder to miss capturing important moments before they pass by.

Even if the technology does not make it to larger sensors, what it brings to the type of sensors used in smartphones clearly demonstrates how far we’ve come since the first models of smartphones arrived to the market: it also explains why we’re seeing so many videographers adopting smartphones for some of their projects, and photographers trust more and more the small cameras on their phones for some picture grabbing. This explains why, having left the photo market, Samsung continues to devote time to imaging R&D, and shows all the investment made in products like the Samsung Galaxy S9, which the company presented as “the camera reimagined”.