Autostereogram - davidar/scholarpedia GitHub Wiki
Autostereogram refers to a single-image form of stereograms that can be free-viewed to achieve a stereoscopic, three-dimensional (3D), depth effect without a stereoscope or any other artificial aids to binocular fusion (see <figref>FurrowsASG1979.jpg</figref>). The principle of the autostereogram is a horizontally repetitive image in which the repeated pattern is modulated in such a way that the image can be viewed with an abnormal convergence or divergence angle to generate the visual impression of a stereoscopic depth image within the space of the pattern. When viewing with normal convergence of the eyes on the physical plane, the image appears only as a flat repeating pattern. When the eyes either converge or diverge at the distance of the pattern repeat, small differences between adjacent pattern cycles provide binocular disparities that are interpreted by the viewer as differences in depth. The disparity structure may be designed to correspond to the depth map of any desired three-dimensional scene, which is perceived when the eyes are held at the appropriate convergence angle. In order to do so, however, the viewer must overcome the natural tendency of the eyes to focus at the convergence distance, and enable the eyes to refocus at the plane of the image.
The history of the autostereogram falls into five phases. The first phase was the early conceptual development, which began with Sir David Brewster in 1844. He observed, and worked out the principles of, the wallpaper illusion. To see this illusion, view a flat wall covered with a wallpaper showing a regular, horizontally repeating pattern then converge the eyes on a point in space closer than the wall. At a certain point, one part of the repeating pattern is imaged in central vision of the left eye and that part's neighbour, immediately to the left, is imaged in central vision of the right eye. Now the visual patterns viewed by the eyes are identical, except for the unpaired parts of the pattern in far peripheral vision, but the convergence angle is closer than the wall. One sees the wallpaper floating in space in front of its true position. Alternatively, one can relax the eyes to diverge them farther from the wall, so that the wallpaper appears to float at a distant location farther than its actual distance.
The second phase was the hand construction of coherent depth images based on systematic shifts in the horizontal position of adjacent elements of the repeated patterns of the wallpaper illusion. This approach seems to have been first used by Masuhiro Ito in 1970, was accurately developed in a modular 3D format by Edward Trent in 1972 (see <figref>Trent1972.jpg</figref>; Trent, 1972), and was patented in 1974 by Donald Peck (Peck, 1974). Geometric pattern autostereogram of a crystalline structure. (Edward Trent, 1972, Bulletin of the Stereoscopic Society.)
The third phase was the development of the algorithm for mapping any arbitrary depth image into a random-dot autostereogram by Christopher Tyler in 1979 in conjunction with programmer Maureen Clarke. This technique, known as the single-image random-dot stereogram (SIRDS), was published by Tyler in a visual science textbook (Tyler, 1983), by David Stork in a computer science compendium (Stork, 1986), and by Dan Dyckman in a computer game magazine (Dyckman, 1990). These computer-generated 3D images used the computerized random-dot stereogram approach, developed by Bela Julesz in 1960, to camouflage the structure of the depth image being represented in the autostereogram when viewed directly. In 1968, Julesz extended this concept to camouflaging the dynamic structure of moving objects by generating stereoscopic movie sequences in which the random-dot pattern was randomly regenerated for every frame to avoid the spatial correlations over time that are formed by physical moving objects (Julesz & Payne, 1968).
The fourth phase of autostereogram development was the explicit, full-color images by Magic Eye and many other commercial companies. These usually required the eyes to overdiverge in order to get the 3D effect, which is rather harder for most people to achieve than overconvergence. In many cases, the designers of these autostereograms abandoned the idea of camouflaging the 2D information and instead organized the repeating patterns into multiple versions of recognisable images (<figref>Hawlisch_HeartAutostereogram.jpg</figref>), often spatially related to the depth image. The requirements of the shifting horizontal positions in adjacent repeats to support the depth image stretch and degrade the recognisable images, especially where there are abrupt changes in disparity within the depth image. There are two basic techniques used with these full-color image base patterns. One is an elaboration of the Trent modular 3D concept, in which the depth structure repeats around the same repetition interval as the base pattern, with progressive variations to provide the requisite binocular disparities. This approach is limited to horizontally repeating 3D structures. The other approach employs the Tyler concept of a 3D structure that is independent of the pattern repeats, and just uses them as a base pattern on which to project the 3D depth map of arbitrary structure. In both cases, 3D images of full complexity and intersecting transparent overlay may be depicted.
The fifth phase of the technique is the generation of dynamic autostereograms, initiated in 1988 by Dan Dyckman with his movie 'Echodots' (<figref>SuperStereoFractalisticClip.gif</figref>). The basic concept is to generate a series of autostereograms depicting successive views of a scene as it evolves in time and string them together to be viewed as a 3D movie without any specialized 3D equipment. However, just as the Julesz random dot stereogram concept masks the perception of the 3D object structure until the stereo image is binocularly fused, the true dynamic autostereogram should be masked to the movement of the underlying repeating pattern as the successive frames appear. To do so, the repeating random-dot pattern is replaced by a new, uncorrelated pattern in each frame. This technique creates the impression of a field of local swirling dots without coherent pattern motion, allowing the motion of the 3D autosterogram structures to be clearly perceived. Dyckman's movie included both in-plane and depth motion of the depicted 3D structures. This technique is known as the Dynamic Uncorrelated Single-Image Random-Dot Stereogram (DUSIRDS)
Subsequent efforts to generate dynamic autostereograms are available in various formats, such as YouTube and animated GIFs, but very few of them employ the full DUSIRDS approach; they either use the same base repetition pattern allowing strong correlated motion cues, or provide additional uniform motion in an attempt to mask the correlated motion cues. This correlated motion can make it difficult to appreciate the 3D structure from the disparity cues. As with static autostereograms, dynamic autostereograms come in two flavors, either the Trent cellular repeating technique or the Tyler independent depth map technique may be employed. In general, it should be noted that human stereopsis is a fine-grain spatial processing system and requires very high-resolution images for optimal effect.
In addition to the random-dot camouflaged autostereogram and the non-camouflaged structured-image autostereogram, Tyler and Clarke (1979) http://www.ski.org/CWTyler_lab/CWTyler/TylerPDFs/TylerClarkeAutosterSPIE1979.pdf described four other types of free-fusion figure deriving from the same concept, although none have gained comparable recognition. Each requires computational tricks to achieve the novel conceptual structure. One is the continuous-depth autostereogram, in which the depth structure cycles around on itself to give an array of overlapping depth structures as the eyes converge on different parts of the image (<figref>CyclicStaircaseAutostereogram.jpg</figref>). Another is the wallpaper autostereogram, with a cyclic pattern that matches at the two ends to allow the infinite tiling of a plane, or wall (rather than the monocular confusion that occurs at the edges of standard autostereograms). A third is the two-way autostereogram, which contains a pair of independent cyclopean depth structures when viewed at rotations 90º apart (<figref>TwoWayAutostereogram.jpg</figref>). The final one is the autolustergram, which contains a cyclopean image depicted in dichoptic luster rather than defined binocular disparities. These latter give a ghostly, evanescent appearance rather than the smooth surfaces of the autostereograms.
This entire section is excerpted with permission from Steven Pinker's How the Mind Works, with minor modifications. Steven Pinker retains copyright. The content in this section is not licensed under any Creative Commons license.
The autostereogram principle is illustrated in <figref>Projection_of_the_eyes_on_an_autostereogram.jpeg</figref> for the simple depth map of a step in depth at the visual midline. The image in the autostereogram plane is indicated by the horizontal series of black dots, representing the repeat interval of the repetitive pattern. In this example, the viewer’s right eye is fixated on the central dot, but the left eye is fixated on the adjacent repeat to the right. This gives a convergence posture of the eyes that yields the perception of false images in front of the autostereogram plane (all the blue dots), in this case a flat plane to the left (left, solid blue line) and a nearer, flat plane to the right (right solid blue line). The dotted lines show all the lines of sight to the black dots, but they also provide a grid of intersections representing all possible locations in space where a black dot could be perceived, due to spurious correspondences between the images of all pairs of black dots. This are all the false matches Kepler realised must occur; the technique of finding them Kepler invented is now referred to as the Keplerian projection field.
For the full autostereogram pattern of dense texture between the repeats, there would be thousands of such intersections in the Keplerian projection field. If our eyes consisted of only the foveas (F), they could fixate at any one of these locations (red lines) and receive the optical information of the presence of a dot in space at that location. The grid of intersections in <figref>Projection_of_the_eyes_on_an_autostereogram.jpeg</figref> shows that the autostereogram generates the optical information for a dense array of possible depth image interpretations both in front of and behind the physical autostereogram plane (such as the blue and yellow dotted planes). The eyes are depicted as focusing at one of these locations away from the physical plane to emphasize that the eyes are free to move anywhere in space in the vicinity of the autostereogram plane, in order to view different aspects of this disparity structure.
Given this superfluity of valid depth locations, how is it that the brain yields perception of the depth structure of just one 3D image? Although there are aspects of this question that still remain to be resolved, the basic explanation may be understood in terms of the spatial projection of the binocular neurons in early visual cortex (Fischer & Poggio, 1977), which have 3D receptive fields or sensitive zones covering small regions of space (diagrammed as the green circles in <figref>Human_physiology_of_stereopsis.jpg</figref>). These 3D sensitive zones lie close to the Vieth-Müller circle of zero disparities keyed to the fixation location (dashed black circle passing through the eyes and the fixation point), indicated by the intersection of the red lines of sight from the two foveas at the back of the eyes (F). Only targets falling within the scope of the 3D receptive field array (green circles) will be perceived as points in depth. This array thus selects out a narrow zone of 3D space in which depth structure may be perceived, and some form of neural interaction mechanism effectively suppresses all other possible targets (Tyler & Kontsevich, 2005). The neural processing system shown in <figref>Human_physiology_of_stereopsis.jpg</figref> should be envisaged as superimposed on the optical array shown the previous figure, selecting one dot at the foveal intersection, and excluding the original autostereogram plane (dashed line). The neural processing also incorporates a mechanism for interpolating a continuous surface through the empty spaces between the dots (Li et al., 2013).
At a far distance, the region of space encompassed by the zones around the Vieth-Müller circle is approximately planar, but at close viewing distances it has noticeable curvature, so it is advantageous to view the autostereograms on a curved surface. This surface is not, however, spherical around the fixation point but cylindrical, with a forward skew passing roughly through the feet (see Tyler, 1991), so it is easy to curve a flat printed autostereogram into the correct cylindrical format for optimal viewing of the depth image throughout the visual field.
The depth-to-change-in-repetition-interval conversion may be achieved for the coloration of any point by setting it to that of the point one repetition interval to the left (or right, depending on how the autostereogram is constructed). The position of that point is determined by the differential shift prescribed by the depth map at that point. (Strictly, it should be the shift prescribed by the depth map at a location in space half of the base repetition interval back from that point.) In this way the full extent of the autostereogram can be built up from the initial vertical strip of seed dots. The only other requirement is to avoid the pattern replication that arises from going to dense repetitions and then switching to sparse ones at an edge. Thus, when the current look-back exceeds the previous one, the overlap should be replaced with new random dots rather than those provided by the standard look-back. The resulting region of unpaired dots has an uncorrelated dot-cloud appearance that is an inevitable result of the binocular viewing of vertical edges, which will always have a region visible to one eye that is hidden from the other due to the binocular viewing geometry.
1792. Charles Wells describes the fusion of the images from the two eyes into a single perceptual image, perceived as being located in the “cyclopean eye” centered halfway between the two physical eyes.
1838. Charles Wheatstone discovers the full properties of binocular disparity in depth perception and invents the first stereoscope, for viewing his dual-image stereograms.
1844. David Brewster rediscovers the wallpaper autostereogram and remarks that defects in the repeats form a disparity-based depth image.
1858. Frenchman Joseph D’Almeida develops the color anaglyph technique for stereo projection from a single image.
1893. William Friese-Greene demonstrates the first 3D movie with an anaglyphic 3D movie camera of his own invention.
1901. Spanish neuroanatomist Santiago Ramon y Cajal invents an early form of random-dot stereogram, as a tool to study the stereoscopic depth processing of binocular vision.
1939. Adelbert Ames III constructs the Leaf Room to camouflage the monocular depth cues to, in order to study the perception of the 3D shape of the room primarily from disparity cues.
1939. In Russia, Boris Kompaneysky publishes the first camouflaged random-blob dual-image stereogram (for the Russian Academy of Fine Arts).
1960. Bela Julesz develops the computerized random-dot dual-image stereogram for encoding any specified disparity profile, perfectly-camouflaged.
1962. Bela Julesz and Joan Miller develop the iterative algorithm for simultaneous encoding of two independent disparity surfaces in a dual-image stereogram. When one of the two surfaces is flat, the depth of the second surface may be seen by free-fusion on one eye’s image alone, forming a random-dot autostereogram. There is no record that Julesz was aware of this single-image capability for viewing the two-surface stereograms.
1968. Bela Julesz and Richard Payne introduce the dynamic stereo movie in random-dot dual-image format and show that stereoscopic features have distinct temporal processing characteristics.
1970. Masayuki Ito uses hand-construction methods to design a fully camouflaged random-dot autostereogram of protruding flat-plane squares.
1972. Edward Trent publishes complex line autostereogram images with continuous depth variation in the Bulletin of the Stereoscopic Society.
1972-3. Roger Ferrogallo develops stereoscopic painting based on a repeating tile structure that generates alternating depth images at the repeat interval.
1974. Roger Ferrogallo publishes an article on his stereoscopic painting technique in /*Leonardo*/.
1974. Alphonse Schilling invents the stereogram triplet for either crossed or uncrossed free-fusion and hand-paints flat-plane autostereograms.
1979. Donald Peck obtains US Patent 4135502 for the block-image autostereogram technique.
1979. Christopher Tyler and Maureen Clarke develop the direct, ‘look-back’ algorithm to convert any specified disparity profile into a random-dot autostereogram.
1980. Peter Burt and Bela Julesz use the line autostereogram method to demonstrate properties of the fusion limit.
1983. First publication of Tyler and Clarke’s autostereograms, generated on an Apple II computer and a dot-matrix printer.
1986. David Stork uses a random-dot autostereogram that he devised with Chris Rocca as the frontispiece of ‘Seeing the Light’ by David Falk, Dieter Brill and David Stork, introducing the technique to the physics and computer science communities.
1988. Dan Dyckman generates the first dynamic autostereogram movie 'Echodots'.
1991. Christopher Tyler and Clarke publish the random-dot autostereogram algorithm, together with two-way autostereograms, depth-cycling autostereograms, and autolustergrams. This paper describes the necessity for inserting uncorrelated dots when increasing the repetition period to correct for the accumulated 'pattern ringing' errors that are found in many later descriptions of the uncorrected algorithm.
1992. Geoffrey Slinker and Robert Burton publish an article on the autostereogram generation that includes the first description of the animation of random dot autostereograms.
1993. Publication of 'Magic Eye: A New Way of Looking at the World' by Tom Baccei, Cheri Smith and Bob Salitsky, the first of an extended series of Magic Eye books.
1994. Dan Dyckman publishes the first autostereogram game book: 'Hidden Dimensions: Use Your Deep Vision to Solve Mazes, Riddles, and Other Perplexing Puzzles'.
1994. Stuart Inglis, Harold Thimbleby and Ian Witten rediscover the Julesz and Miller (1962) algorithm for generating autostereograms without the accumulated 'pattern ringing' errors of the uncorrected look-back algorithm.
1994. Publication of ‘CG Stereogram’, edited by Seiji Horibuchi, with a variety of high-resolution, full color and image-based autostereograms and extended analysis of their historical development by Christopher Tyler and by Itsuo Sakane.
2001. Harry Potter Magic Eye autostereogram books available in Germany and many other countries.
Brewster, D. (1844) On the knowledge of distance given by binocular vision. Trans. Royal Soc. Edin. 15, 663-674.
Burt, P., Julesz, B. (1980) A disparity gradient limit for binocular fusion. Science, 208, 615-617.
Dyckman, D. (1994) Hidden Dimensions: Use Your Deep Vision to Solve Mazes, Riddles, and Other Perplexing Puzzles. New York: Harmony Books.
Ferrogallo, R. (1974) On stereoscopic painting. Leonardo, 7, 97-104.
Poggio G.F., Fischer B. (1977) Binocular interaction and depth sensitivity in striate and prestriate cortex of behaving rhesus monkey. J Neurophysiol. 40(6):1392-405.
Horibuchi, S. (1994) (Ed.), CG Stereogram. San Francisco: Cadence Books.
Inglis, S., Thimbleby, H., Witten, I.H. (1994) "Displaying 3D Images: Algorithms for Single-Image Random-Dot Stereograms," IEEE Computer, 27(10), pp.38-48.
Julesz, B. (1960) Binocular depth perception for computer generated patterns. Bell Syst. Tech. J. 39, 1125-1162.
Julesz, B., Miller, J.E. (1962). Automatic stereoscopic presentation of functions of two variables. Bell Syst. Tech. J. 41:663–676.
Julesz, B., Payne, R.A. (1968) Differences between monocular and binocular stroboscopic movement perception, Vision Research 8, 433-44.
Li, X., Huang, A.E., Altschuler, E.L., Tyler, C.W. (2013) Depth spreading through empty space induced by sparse disparity cues. J Vis. 13(10). pii: 7.
Magic Eye Inc. (1993) Magic eye: A new way of looking at the world : 3D illusions. Boston: NE Thing Enterprises.
Panini Books (2001). Harry Potter, Magische Bilder (Gebundene Ausgabe).
Peck, D. (1979) Stereoscopic patterns and method of making same. US Patent 4135502.
Pepin, S. (2003) Adelbert Ames and the Dartmouth Eye Institute. J. Neuro-Ophthal. 23, 290-297.
Pinker, S. (1997). The mind's eye. In Pinker S., How the Mind Works. pp. 211–233.
Ramon y Cahal, S. (1901) Recreaciones estereoscópicas y binoculares. La Fotografia, 27, 41-8.
Sakane, I. (1994). In Horibuchi, S. (Ed.), CG Stereogram. San Francisco: Cadence Books (pp. 38-45).
Slinker G.S., Burton, R.P. (1992) The generation and animation of random dot autostereograms. J. Imaging Science and Technology, 36(3). 260-267.
Stork, D.G., Rocca, C. (1989) Software for generating auto-random-dot stereograms, Behavior Research Methods, Instruments, & Computers 21, 525-534.
Trent, E. (1972) Stereo designs as an art form. Bulletin of the Stereoscopic Society, 37, 7-10.
Tyler, C.W., Clarke, M.B. (1990) The autostereogram. Stereoscopic Displays and Applications, Proc. SPIE Vol. 1258:182–196.
Tyler, C.W. (1994). Random dot stereogram 2. In Horibuchi, S. (Ed.), CG Stereogram. San Francisco: Cadence Books (pp. 29-34).
Tyler, C.W. (1994). The birth of computer stereograms for unaided stereovision. In Horibuchi, S. (Ed.), CG Stereogram. San Francisco: Cadence Books (pp. 83–89).
Tyler, C.W., Kontsevich, L.L (2005) The structure of stereoscopic masking: Position, disparity, and size tuning. Vision Res. 43, 3096-3108.
Wade, N. J. (2003) Destined for Distinguished Oblivion: The Scientific Vision of William Charles Wells (1757-1817). New York: Kluwer/Plenum.