Biologically-inspired Computer Vision: Fundamentals and Applications

As the state–of–the–art imaging technologies became more and more advanced, yielding scientific data at unprecedented detail and volume, the need to process and interpret all the data has made image processing and computer vision increasingly important. Sources of data that have to be routinely dealt with today?s applications include video transmission, wireless communication, automatic fingerprint processing, massive databanks, non–weary and accurate automatic airport screening, robust night vision, just to name a few. Multidisciplinary inputs from other disciplines such as physics, computational neuroscience, cognitive science, mathematics, and biology will have a fundamental impact in the progress of imaging and vision sciences. One of the advantages of the study of biological organisms is to devise very different type of computational paradigms by implementing a neural network with a high degree of local connectivity. This is a comprehensive and rigorous reference in the area of biologically motivated vision sensors. The study of biologically visual systems can be considered as a two way avenue. On the one hand, biological organisms can provide a source of inspiration for new computational efficient and robust vision models and on the other hand machine vision approaches can provide new insights for understanding biological visual systems. Along the different chapters, this book covers a wide range of topics from fundamental to more specialized topics, including visual analysis based on a computational level, hardware implementation, and the design of new more advanced vision sensors. The last two sections of the book provide an overview of a few representative applications and current state of the art of the research in this area. This makes it a valuable book for graduate, Master, PhD students and also researchers in the field. INDICE: List of Contributors XV .Foreword XIX .Part I Fundamentals 1 .1 Introduction 3Gabriel Cristóbal, Laurent U. Perrinet, and Matthias S. Keil .1.1 Why ShouldWe Be Inspired by Biology? 4 .1.2 Organization of Chapters in the Book 6 .1.3 Conclusion 9 .Acknowledgments 9 .References 9 .2 Bioinspired Vision Sensing 11Christoph Posch .2.1 Introduction 11 .2.1.1 Neuromorphic Engineering 12 .2.1.2 Implementing Neuromorphic Systems 13 .2.2 Fundamentals and Motivation: Bioinspired Artificial Vision 13 .2.2.1 Limitations in Vision Engineering 14 .2.2.2 The Human Retina from an Engineering Viewpoint 14 .2.2.3 Modeling the Retina in Silicon 17 .2.3 From Biological Models to Practical Vision Devices 18 .2.3.1 TheWiring Problem 18 .2.3.2 Where and What 20 .2.3.3 Temporal Contrast: The DVS 21 .2.3.4 Event–Driven Time–Domain Imaging: The ATIS 22 .2.4 Conclusions and Outlook 25 .References 26 .3 Retinal Processing: From Biology to Models and Applications 29David Alleysson and Nathalie Guyader .3.1 Introduction 29 .3.2 Anatomy and Physiology of the Retina 30 .3.2.1 Overview of the Retina 30 .3.2.2 Photoreceptors 31 .3.2.3 Outer and Inner Plexiform Layers (OPL and IPL) 33 .3.2.4 Summary 34 .3.3 Models of Vision 34 .3.3.1 Overview of the Retina Models 34 .3.3.2 Biological Models 35 .3.3.2.1 Cellular and Molecular Models 35 .3.3.2.2 Network Models 36 .3.3.2.3 Parallel and Descriptive Models 38 .3.3.3 Information Models 39 .3.3.4 Geometry Models 40 .3.4 Application to Digital Photography 42 .3.4.1 Color Demosaicing 43 .3.4.2 Color Constancy Chromatic Adaptation White Balance Tone Mapping 44 .3.5 Conclusion 45 .References 46 .4 Modeling Natural Image Statistics 53Holly E. Gerhard, Lucas Theis, and Matthias Bethge .4.1 Introduction 53 .4.2 Why Model Natural Images? 53 .4.3 Natural Image Models 55 .4.3.1 Model Evaluation 65 .4.4 Computer Vision Applications 69 .4.5 Biological Adaptations to Natural Images 71 .4.6 Conclusions 75 .References 76 .5 Perceptual Psychophysics 81C. Alejandro Parraga .5.1 Introduction 81 .5.1.1 What Is Psychophysics andWhy DoWe Need It? 81 .5.2 Laboratory Methods 82 .5.2.1 Accuracy and Precision 83 .5.2.2 Error Propagation 84 .5.3 PsychophysicalThreshold Measurement 85 .5.3.1 Weber s Law 85 .5.3.2 Sensitivity Functions 86 .5.4 Classic Psychophysics:Theory and Methods 86 .5.4.1 Theory 87 .5.4.2 Method of Constant Stimuli 88 .5.4.3 Method of Limits 90 .5.4.3.1 Forced–Choice Methods 92 .5.4.4 Method of Adjustments 93 .5.4.5 Estimating Psychometric Function Parameters 94 .5.5 Signal Detection Theory 94 .5.5.1 Signal and Noise 94 .5.5.2 The Receiver Operating Characteristic 96 .5.6 Psychophysical Scaling Methods 98 .5.6.1 Discrimination Scales 99 .5.6.2 Rating Scales 100 .5.6.2.1 Equipartition Scales 100 .5.6.2.2 Paired Comparison Scales 101 .5.7 Conclusions 105 .References 106 .Part II Sensing 109 .6 Bioinspired Optical Imaging 111Mukul Sarkar .6.1 Visual Perception 111 .6.1.1 Natural Single–Aperture and Multiple–Aperture Eyes 112 .6.1.1.1 Human Eyes 113 .6.1.1.2 Compound Eyes 114 .6.1.1.3 Resolution 114 .6.1.1.4 Visual Acuity 115 .6.1.1.5 Depth Perception 115 .6.1.1.6 Color Vision 116 .6.1.1.7 Speed of Imaging and Motion Detection 117 .6.1.1.8 Polarization Vision 117 .6.2 Polarization Vision – Object Differentiation/Recognition 119 .6.2.1 Polarization of Light 121 .6.2.2 Polarization Imaging 125 .6.2.2.1 Detection of Transparent and Opaque Objects 126 .6.2.2.2 Shape Detection Using Polarized Light 131 .6.3 High–Speed Motion Detection 133 .6.3.1 Temporal Differentiation 135 .6.4 Conclusion 138 .References 139 .7 Biomimetic Vision Systems 143Reinhard Voelkel .7.1 Introduction 143 .7.2 Scaling Laws in Optics 144 .7.2.1 Optical Properties of Imaging Lens Systems 144 .7.2.2 Space Bandwidth Product (SW) 146 .7.3 The Evolution of Vision Systems 147 .7.3.1 Single–Aperture Eyes 148 .7.3.2 Compound Eyes 149 .7.3.3 The Array Optics Concept 150 .7.3.4 Jumping Spiders: Perfect Eyes for Small Animals 151 .7.3.5 Nocturnal Spiders: Night Vision 153 .7.4 Manufacturing of Optics for Miniaturized Vision Systems 154 .7.4.1 Optics Industry 154 .7.4.2 Planar Array Optics for Stereoscopic Vision 155 .7.4.3 The Lack of a Suitable Fabrication Technology Hinders Innovation 155 .7.4.4 Semiconductor Industry PromotesWafer–Based Manufacturing 156 .7.4.5 Image Sensors 156 .7.4.6 Wafer–Based Manufacturing of Optics 158 .7.4.7 Manufacturing of Microlens Arrays onWafer Level 159 .7.4.8 Diffractive Optical Elements and Subwavelength Structures for Antireflection Coatings 160 .7.4.9 Microlens Imprint and Replication Processes 162 .7.4.10 Wafer–Level Stacking or Packaging (WLP) 162 .7.4.11 Wafer–Level Camera (WLC) 163 .7.5 Examples for Biomimetic Compound Vision Systems 164 .7.5.1 Ultraflat Cameras 165 .7.5.2 Biologically Inspired Vision Systems for Smartphone Cameras 166 .7.5.3 PiCam Cluster Camera 167 .7.5.4 Panoramic Motion Camera for Flying Robots 169 .7.5.5 Conclusion 170 .References 172 .8 Plenoptic Cameras 175Fernando Pérez Nava, Alejandro Pérez Nava, Manuel Rodríguez Valido, and Eduardo Magdaleno Castellò .8.1 Introduction 175 .8.2 Light Field Representation of the Plenoptic Function 177 .8.2.1 The Plenoptic Function 177 .8.2.2 Light Field Parameterizations 178 .8.2.3 Light Field Reparameterization 179 .8.2.4 Image Formation 180 .8.2.5 Light Field Sampling 180 .8.2.5.1 Pinhole and Thin Lens Camera 180 .8.2.5.2 Multiple Devices 181 .8.2.5.3 Temporal Multiplexing 181 .8.2.5.4 Frequency Multiplexing 182 .8.2.5.5 Spatial Multiplexing 182 .8.2.5.6 Simulation 182 .8.2.6 Light Field Sampling Analysis 183 .8.2.7 Light Field Visualization 183 .8.3 The Plenoptic Camera 185 .8.4 Applications of the Plenoptic Camera 188 .8.4.1 Refocusing 188 .8.4.2 Perspective Shift 190 .8.4.3 Depth Estimation 190 .8.4.4 Extended Depth of Field Images 192 .8.4.5 Superresolution 192 .8.5 Generalizations of the Plenoptic Camera 193 .8.6 High–Performance Computing with Plenoptic Cameras 195 .8.7 Conclusions 196 .References 197 .Part III Modelling 201 .9 Probabilistic Inference and Bayesian Priors in Visual Perception 203Grigorios Sotiropoulos and Peggy Seriès .9.1 Introduction 203 .9.2 Perception as Bayesian Inference 204 .9.2.1 Deciding on a Single Percept 205 .9.3 Perceptual Priors 207 .9.3.1 Types of Prior Expectations 207 .9.3.2 Impact of Expectations 208 .9.3.3 The Slow Speed Prior 210 .9.3.4 Expectations and Environmental Statistics 212 .9.4 Outstanding Questions 214 .9.4.1 Are Long–Term Priors Plastic? 214 .9.4.2 How Specific are Priors? 215 .9.4.3 Inference in Biological and Computer Vision 215 .9.4.4 Conclusions 217 .References 219 .10 From NeuronalModels to Neuronal Dynamics and Image Processing 221Matthias S. Keil .10.1 Introduction 221 .10.2 The Membrane Equation as a Neuron Model 222 .10.2.1 Synaptic Inputs 224 .10.2.2 Firing Spikes 227 .10.3 Application 1: A Dynamical Retinal Model 230 .10.4 Application 2: Texture Segregation 234 .10.5 Application 3: Detection of Collision Threats 236 .10.6 Conclusions 239 .Acknowledgments 240 .References 240 .11 ComputationalModels of Visual Attention and Applications 245Olivier LeMeur and Matei Mancas .11.1 Introduction 245 .11.2 Models of Visual Attention 246 .11.2.1 Taxonomy 246 .11.3 A Closer Look at Cognitive Models 250 .11.3.1 Itti et al. s Model [3] 250 .11.3.2 Le Meur et al. s Model [8] 251 .11.3.2.1 Motivations 251 .11.3.2.2 Global Architecture 252 .11.3.3 Limitations 255 .11.4 Applications 256 .11.4.1 Saliency–Based Applications: A Brief Review 256 .11.4.2 Predicting Memorability of Pictures 257 .11.4.2.1 Memorability Definition 257 .11.4.2.2 Memorability and Eye Movement 257 .11.4.2.3 Computational Models 259 .11.4.3 Quality Metric 260 .11.4.3.1 Introduction 260 .11.4.3.2 Eye Movement During a Quality Task 260 .11.4.3.3 Saliency–Based Quality Metrics 261 .11.5 Conclusion 262 .References 262 .12 Visual Motion Processing and Human Tracking Behavior 267Anna Montagnini, Laurent U. Perrinet, and Guillaume S. Masson .12.1 Introduction 267 .12.2 Pursuit Initiation: Facing Uncertainties 269 .12.2.1 Where Is the Noise? Motion–Tracking Precision and Accuracy 269 .12.2.2 Where Is the Target Really Going? 270 .12.2.3 Human Smooth Pursuit as Dynamic Readout of the Neural Solution to the Aperture Problem 272 .12.3 Predicting Future and On–Going Target Motion 273 .12.3.1 Anticipatory Smooth Tracking 273 .12.3.2 If You Don t See It, You Can Still Predict (and Track) It 274 .12.4 Dynamic Integration of Retinal and Extra–Retinal Motion Information: Computational Models 276 .12.4.1 A Bayesian Approach for Open–Loop Motion Tracking 276 .12.4.2 Bayesian (or Kalman–Filtering) Approach for Smooth Pursuit: Hierarchical Models 278 .12.4.3 A Bayesian Approach for Smooth Pursuit: Dealing with Delays 279 .12.5 Reacting, Inferring, Predicting: A NeuralWorkspace 282 .12.6 Conclusion 286 .12.6.1 Interest for Computer Vision 287 .Acknowledgments 288 .References 288 .13 Cortical Networks of Visual Recognition 295Christian Thériault, Nicolas Thome, and Matthieu Cord .13.1 Introduction 295 .13.2 Global Organization of the Visual Cortex 296 .13.3 Local Operations: Receptive Fields 297 .13.4 Local Operations in V1 298 .13.5 Multilayer Models 301 .13.6 A Basic IntroductoryModel 302 .13.7 Idealized Mathematical Model of V1: Fiber Bundle 307 .13.8 Horizontal Connections and the Association Field 311 .13.9 Feedback and Attentional Mechanisms 312 .13.10 Temporal Considerations, Transformations and Invariance 312 .13.11 Conclusion 314 .References 315 .14 Sparse Models for Computer Vision 319Laurent U. Perrinet .14.1 Motivation 319 .14.1.1 Efficiency and Sparseness in Biological Representations of Natural Images 319 .14.1.2 Sparseness Induces Neural Organization 320 .14.1.3 Outline: Sparse Models for Computer Vision 322 .14.2 What Is Sparseness? Application to Image Patches 323 .14.2.1 Definitions of Sparseness 323 .14.2.2 Learning to Be Sparse:The SparseNet Algorithm 325 .14.2.3 Results: Efficiency of Different Learning Strategies 326 .14.3 SparseLets: A Multiscale, Sparse, Biologically Inspired Representation of Natural Images 328 .14.3.1 Motivation: Architecture of the Primary Visual Cortex 328 .14.3.2 The SparseLets Framework 330 .14.3.3 Efficiency of the SparseLets Framework 333 .14.4 SparseEdges: Introducing Prior Information 336 .14.4.1 Using the Prior in First–Order Statistics of Edges 336 .14.4.2 Using the Prior Statistics of Edge Co–Occurrences 338 .14.5 Conclusion 341 .Acknowledgments 341 .References 342 .15 Biologically Inspired Keypoints 347Alexandre Alahi, Georges Goetz, and Emmanuel D Angelo .15.1 Introduction 347 .15.2 Definitions 349 .15.3 What Does the Frond–End of the Visual System Tell Us? 350 .15.3.1 The Retina 350 .15.3.2 From Photoreceptors to Pixels 350 .15.3.3 Visual Compression 351 .15.3.4 Retinal Sampling Pattern 351 .15.3.5 Scale–Space Representation 352 .15.3.6 Difference of Gaussians as a Model for RGC–Receptive Fields 353 .15.3.7 A Linear Nonlinear Model 354 .15.3.8 Gabor–Like Filters 354 .15.4 Bioplausible Keypoint Extraction 355 .15.4.1 Scale–Invariant Feature Transform 355 .15.4.2 Speeded–Up Robust Features 356 .15.4.3 Center Surround Extrema 356 .15.4.4 Features from Accelerated Segment Test 357 .15.5 Biologically Inspired Keypoint Representation 357 .15.5.1 Motivations 357 .15.5.2 Dense Gabor–Like Descriptors 358 .15.5.2.1 Scale–Invariant Feature Transform Descriptor 358 .15.5.2.2 Speeded–Up Robust Features Descriptor 359 .15.5.3 Sparse Gaussian Kernels 360 .15.5.3.1 Local Binary Descriptors 360 .15.5.3.2 Fast Retina Keypoint Descriptor 360 .15.5.3.3 Fast Retina Keypoint Saccadic Matching 362 .15.5.4 Fast Retina Keypoint versus Other Local Binary Descriptors 363 .15.6 Qualitative Analysis: Visualizing Keypoint Information 363 .15.6.1 Motivations 363 .15.6.2 Binary Feature Reconstruction: From Bits to Image Patches 364 .15.6.2.1 Feature Inversion as an Inverse Problem 364 .15.6.2.2 Interest of the Retinal Descriptor 367 .15.6.3 From Feature Visualization to Crowd–Sourced Object Recognition 368 .15.7 Conclusions 370 .References 371 .Part IV Applications 375 .16 Nightvision Based on a Biological Model 377Magnus Oskarsson, Henrik Malm, and EricWarrant .16.1 Introduction 377 .16.1.1 RelatedWork 378 .16.2 Why Is Vision Difficult in Dim Light? 380 .16.3 Why Is Digital Imaging Difficult in Dim Light? 382 .16.4 Solving the Problem of Imaging in Dim Light 383 .16.4.1 Enhancing the Image 385 .16.4.1.1 Visual Image Enhancement in the Retina 385 .16.4.1.2 Digital Image Enhancement 387 .16.4.2 Filtering the Image 388 .16.4.2.1 Spatial and Temporal Summation in Higher Visual Processing 388 .16.4.2.2 Structure Tensor Filtering of Digital Images 391 .16.5 Implementation and Evaluation of the Night–Vision Algorithm 393 .16.5.1 Adaptation of Parameters to Noise Levels 394 .16.5.2 Parallelization and Computational Aspects 395 .16.5.3 Considerations for Color 396 .16.5.4 Experimental Results 397 .16.6 Conclusions 399 .Acknowledgment 400 .References 401 .17 Bioinspired Motion Detection Based on an FPGA Platform 405Tim Köhler .17.1 Introduction 405 .17.2 A Motion Detection Module for Robotics and Biology 406 .17.3 Insect Motion Detection Models 407 .17.4 Overview of Robotic Implementations of Bioinspired Motion Detection 412 .17.4.1 Field–Programmable Gate Arrays (FPGA) 413 .17.5 An FPGA–Based Implementation 414 .17.5.1 FPGA–Camera–Module 414 .17.5.2 A Configurable Array of EMDs 416 .17.6 Experimental Results 419 .17.7 Discussion 421 .17.8 Conclusion 422 .Acknowledgments 423 .References 423 .18 Visual Navigation in a ClutteredWorld 425N. Andrew Browning and Florian Raudies .18.1 Introduction 425 .18.2 Cues from Optic Flow: Visually Guided Navigation 426 .18.3 Estimation of Self–Motion: Knowing Where You Are Going 429 .18.4 Object Detection: Understanding What Is in YourWay 434 .18.5 Estimation of TTC: Time Constraints from the Expansion Rate 439 .18.6 Steering Control: The Importance of Representation 442 .18.7 Conclusions 444 .Acknowledgments 445 .References 445 .Index 447

• ISBN: 978-3-527-41264-8
• Editorial: Wiley VCH
• Encuadernacion: Cartoné
• Páginas: 480
• Fecha Publicación: 14/10/2015
• Nº Volúmenes: 1
• Idioma: Inglés