Core Image
Encyclopedia
Core Image is a pixel-accurate, near-realtime, non-destructive image processing technology in Mac OS X. Implemented as part of the QuartzCore framework of Mac OS X 10.4 and later, Core Image provides a plugin-based architecture for applying filters and effects within the Quartz
graphics rendering layer.
Like Photoshop, Core Image can apply multiple filters to the same image source. Instead of applying a series of filters individually, Core Image assembles a dynamic instruction pipeline so that only one calculation needs to be applied to the pixel data to achieve a cumulative effect. Applying the pixel operations associated with multiple filters can be achieved simultaneously and without a significant increase in processing time. Regardless of the number of filters, Core Image assembles the code for this instruction pipeline with a just-in-time compiler
, which is executed by either the CPU or graphics card's GPU, whichever can perform the calculation faster.
Filters are written in the Core Image Kernel Language, which shares a subset of commands with OpenGL Shading Language (GLSL). When a compatible GPU is available, the Core Image compiler writes the instruction pipeline using GLSL, handling buffers and states transparently. Although GPU rendering is preferred, the compiler can operate in a CPU fallback mode, generating commands suitable for the current CPU architecture instead. CPU fallback uses the vector processing capabilities of the current CPU or CPUs, and it is multi-processor aware. Thus, Core Image performance depends on the GLSL capabilities of the GPU or the processing power of the CPU. With a supported GPU, most effects can be rendered in realtime or near-realtime.
.
Early examples of its use can be found in the ripple effect in Dashboard, and Apple's professional digital photography application, Aperture
. Starting with Mac OS X 10.5
, any application that implements Image Kit can utilize Core Image. Preview
and iPhoto
are common examples.
The Xcode
Tools include Core Image Fun House and Quartz Composer
; both utilize Core Image.
) color space, storing four color channels: red, green, blue, and transparency (alpha). Each color channel is represented by a 32-bit, floating point
number. This provides exceptional color depth, far greater than can be perceived by the human eye
, as each pixel is represented by a 128-bit vector (four 32-bit color channels). For color spaces of lower bit-depth, the floating-point calculation model employed by Core Image provides exceptional performance, which is useful when processing multiple images or video frames.
) commands is Core Image capable. Apple has used the following graphics cards to support Core Image GPU processing in Mac OS X 10.4 and Aperture, so the following list could be considered an example of minimum requirements:
Note that any GPU capable of handling Core Image instructions is also Quartz Extreme
capable. The requirements for Core Image are greater than those of Quartz Extreme.
Mac OS X includes many built-in filters, including the following ones. Mac OS X 10.4 introduced over 100 of them, and Mac OS X 10.5 has added to the list.
Quartz (graphics layer)
Quartz specifically refers to a pair of Mac OS X technologies, each part of the Core Graphics framework: Quartz 2D and Quartz Compositor. It includes both a 2D renderer in Core Graphics and the composition engine that sends instructions to the graphics card...
graphics rendering layer.
Overview
Core Image abstracts the pixel-level manipulation process required when applying a filter to an image, making it simple for applications to implement image transformation capabilities without extensive coding. In a simple implementation, Core Image applies a single Image Filter to the pixel data of a given source to produce the transformed image. Each Image Filter specifies a single transform or effect, either built into Core Image or loaded from a plugin called an Image Unit. Combined with preset or user-defined input parameters, the filter can be applied to the original pixel data without modifying it, thereby providing non-destructive image editing.Like Photoshop, Core Image can apply multiple filters to the same image source. Instead of applying a series of filters individually, Core Image assembles a dynamic instruction pipeline so that only one calculation needs to be applied to the pixel data to achieve a cumulative effect. Applying the pixel operations associated with multiple filters can be achieved simultaneously and without a significant increase in processing time. Regardless of the number of filters, Core Image assembles the code for this instruction pipeline with a just-in-time compiler
Just-in-time compilation
In computing, just-in-time compilation , also known as dynamic translation, is a method to improve the runtime performance of computer programs. Historically, computer programs had two modes of runtime operation, either interpreted or static compilation...
, which is executed by either the CPU or graphics card's GPU, whichever can perform the calculation faster.
Filters are written in the Core Image Kernel Language, which shares a subset of commands with OpenGL Shading Language (GLSL). When a compatible GPU is available, the Core Image compiler writes the instruction pipeline using GLSL, handling buffers and states transparently. Although GPU rendering is preferred, the compiler can operate in a CPU fallback mode, generating commands suitable for the current CPU architecture instead. CPU fallback uses the vector processing capabilities of the current CPU or CPUs, and it is multi-processor aware. Thus, Core Image performance depends on the GLSL capabilities of the GPU or the processing power of the CPU. With a supported GPU, most effects can be rendered in realtime or near-realtime.
History & Implementation
Core Image was introduced with Mac OS X 10.4Mac OS X v10.4
Mac OS X v10.4 Tiger is the fifth major release of Mac OS X, Apple's desktop and server operating system for Macintosh computers. Tiger was released to the public on 29 April 2005 for US$129.95 as the successor to Mac OS X Panther , which had been released 18 months earlier...
.
Early examples of its use can be found in the ripple effect in Dashboard, and Apple's professional digital photography application, Aperture
Aperture (photography software)
Aperture is a photo editing and management software program that was developed by Apple for the Mac OS X operating system, first released in 2005 for $499 , dropped to $199, and now released on their App Store for $79...
. Starting with Mac OS X 10.5
Mac OS X v10.5
Mac OS X Leopard is the sixth major release of Mac OS X, Apple's desktop and server operating system for Macintosh computers. Leopard was released on 26 October 2007 as the successor of Tiger , and is available in two variants: a desktop version suitable for personal computers, and a...
, any application that implements Image Kit can utilize Core Image. Preview
Preview (software)
Preview is Mac OS X's application for displaying images and Portable Document Format documents. Like Mac OS X itself, it comes from NeXT's OPENSTEP operating system....
and iPhoto
IPhoto
iPhoto is a digital photograph manipulation software application developed by Apple Inc. and released with every Macintosh personal computer as part of the iLife suite of digital life management applications...
are common examples.
The Xcode
Xcode
Xcode is a suite of tools, developed by Apple, for developing software for Mac OS X and iOS. Xcode 4.2, the latest major version, is available on the Mac App Store for free for Mac OS X 10.7 , and on the Apple Developer Connection website for free to registered developers Xcode is a suite of tools,...
Tools include Core Image Fun House and Quartz Composer
Quartz Composer
Quartz Composer is a node-based visual programming language provided as part of the Xcode development environment in Mac OS X for processing and rendering graphical data....
; both utilize Core Image.
Pixel Accuracy
All pixel processing provided by an Image Unit is performed in a pre-multiplied alpha (RGBARGBA color space
RGBA stands for Red Green Blue Alpha. While it is sometimes described as a color space, it is actually simply a use of the RGB color model, with extra information. The color is RGB, and may belong to any RGB color space, but an integral alpha value as invented by Catmull and Smith between 1971 and...
) color space, storing four color channels: red, green, blue, and transparency (alpha). Each color channel is represented by a 32-bit, floating point
Floating point
In computing, floating point describes a method of representing real numbers in a way that can support a wide range of values. Numbers are, in general, represented approximately to a fixed number of significant digits and scaled using an exponent. The base for the scaling is normally 2, 10 or 16...
number. This provides exceptional color depth, far greater than can be perceived by the human eye
Human eye
The human eye is an organ which reacts to light for several purposes. As a conscious sense organ, the eye allows vision. Rod and cone cells in the retina allow conscious light perception and vision including color differentiation and the perception of depth...
, as each pixel is represented by a 128-bit vector (four 32-bit color channels). For color spaces of lower bit-depth, the floating-point calculation model employed by Core Image provides exceptional performance, which is useful when processing multiple images or video frames.
Supported Graphics Processors
Any programmable GPU that supports the required Open GL Shader (GLSLGLSL
OpenGL Shading Language , is a high-level shading language based on the syntax of the C programming language...
) commands is Core Image capable. Apple has used the following graphics cards to support Core Image GPU processing in Mac OS X 10.4 and Aperture, so the following list could be considered an example of minimum requirements:
- ATI Mobility Radeon 9700 (Mobility Radeon 9600 is also capable)
- ATI Radeon 9550, 9600, 9650, 9600 XT, 9600 Pro, 9700 Pro, 9800 XT, and 9800 Pro
- ATI Radeon X600 XT, X600 Pro, X800 XT, X850 XT, X1600, X1900 XT
- ATI Radeon HD 2400 XT, HD 2600 Pro, and HD 2600 XT
- NVIDIA GeForce FX 5200 Ultra, FX Go 5200
- NVIDIA GeForce 6600, 6600 LE, 6800 Ultra DDL, and 6800 GT DDL
- NVIDIA GeForce 7300 GT, 7600 GT, and 7800 GT
- NVIDIA GeForce 8600M GT & 8800 GT
- NVIDIA GeForce 9400M & 9600M GT
- NVIDIA GeForce GT 120 & GT 130
- NVIDIA GeForce GTX 285
- NVIDIA Quadro FX 4500 & FX 5600
- NVIDIA Quadro FX 4800
- NVIDIA Quadro 4000
- NVIDIA GeForce 320M
- NVIDIA GeForce GT 330M
- Intel GMA 900, GMA 950, or GMA X3100
Note that any GPU capable of handling Core Image instructions is also Quartz Extreme
Quartz Compositor
Quartz Compositor is the windowing system in Mac OS X. It is responsible for presenting and maintaining rasterized, rendered graphics from the rest of the Core Graphics framework and other renderers in the Quartz technologies family...
capable. The requirements for Core Image are greater than those of Quartz Extreme.
Built-in Filters
- Blurs, like Median, Gaussian, Motion, and Noise
- Color Adjustments & Effects, like Exposure, Gamma Adjust, and Sepia Tone
- Composition Operations, like Addition and Multiply Blends, or Hard Light
- Distortions, like Pinch, Circle Splash, and Vortex
- Generators, like Star Shine and Lenticular Halo
- Geometry Adjustments, such as Crop, Scale, Rotate, and Affine transformationAffine transformationIn geometry, an affine transformation or affine map or an affinity is a transformation which preserves straight lines. It is the most general class of transformations with this property...
- Halftone, Tile, and Posterize filters
- Transitions, such as Dissolve and Ripple
See also
- ApertureAperture (photography software)Aperture is a photo editing and management software program that was developed by Apple for the Mac OS X operating system, first released in 2005 for $499 , dropped to $199, and now released on their App Store for $79...
- Core Image Fun House
- Quartz ComposerQuartz ComposerQuartz Composer is a node-based visual programming language provided as part of the Xcode development environment in Mac OS X for processing and rendering graphical data....
- NodeBox
- Naked lightNaked lightNaked light is a non-destructive image editor for Mac OS X, featuring node-based compositing, infinite resolution, and live filters.Naked light is currently in public beta.- Features :* Non-destructive workflow* Node-based compositing* Infinite resolution...
- PixelmatorPixelmatorPixelmator is a graphic editor developed for Mac OS X, by Pixelmator Team Ltd built upon a combination of open source and Mac OS X technologies...
- impromptuImpromptu (programming environment)Impromptu is a Mac OS X programming environment for live coding. Impromptu is built around the Scheme language, which is a member of the Lisp family of languages...
- GEGLGEGLThe Generic Graphics Library is a programming library under development for image processing applications. It is mainly developed for GIMP in order to bring support for higher bit depth images than GIMP currently supports, as well as non-destructive editing...
, a programming library used by GIMPGIMPGIMP is a free software raster graphics editor. It is primarily employed as an image retouching and editing tool and is freely available in versions tailored for most popular operating systems including Microsoft Windows, Apple Mac OS X, and Linux.In addition to detailed image retouching and...
External links
- Apple Previews Mac OS X “Tiger” - announcement of Mac OS X 10.4, including Core Image
- Developing with Core Image - introduction to Core Image programming from Apple