{"id":72615,"date":"2023-06-28T22:09:10","date_gmt":"2023-06-28T21:09:10","guid":{"rendered":"https:\/\/smartmileco.com\/researchers-reconstruct-3d-environments-from-eye-reflections\/"},"modified":"2023-06-28T22:09:10","modified_gmt":"2023-06-28T21:09:10","slug":"researchers-reconstruct-3d-environments-from-eye-reflections","status":"publish","type":"post","link":"https:\/\/smartmileco.com\/researchers-reconstruct-3d-environments-from-eye-reflections\/","title":{"rendered":"Researchers reconstruct 3D environments from eye reflections","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

<\/p>\n

\n

Researchers at the University of Maryland have turned eye reflections into (somewhat discernible) 3D scenes. The work builds on Neural Radiance Fields (NeRF), an AI technology that can reconstruct environments from 2D photos. Although the eye-reflection approach has a long way to go before it spawns any practical applications, the study<\/a> (first reported by<\/a> Tech Xplore<\/em>) provides a fascinating glimpse into a technology that could eventually reveal an environment from a series of simple portrait photos.<\/p>\n

The team used subtle reflections of light captured in human eyes (using consecutive images shot from a single sensor) to try to discern the person\u2019s immediate environment. They began with several high-resolution images from a fixed camera position, capturing a moving individual looking toward the camera. They then zoomed in on the reflections, isolating them and calculating where the eyes were looking in the photos.<\/p>\n

<\/p>\n

The results (here\u2019s the entire set animated<\/a>) show a decently discernible environmental reconstruction from human eyes in a controlled setting. A scene captured using a synthetic eye (below) produced a more impressive dreamlike scene. However, an attempt to model eye reflections from Miley Cyrus and Lady Gaga music videos only produced vague blobs that the researchers could only guess were an LED grid and a camera on a tripod \u2014 illustrating how far the tech is from real-world use.<\/p>\n

\"A
Reconstructions using a synthetic eye were much more vivid and lifelike \u2014 with a dreamlike quality.<\/figcaption>

University of Maryland<\/p>\n<\/figure>\n

The team overcame significant obstacles to reconstruct even crude and fuzzy scenes. For example, the cornea introduces \u201cinherent noise\u201d that makes it difficult to separate the reflected light from humans\u2019 complex iris textures. To address that, they introduced cornea pose optimization (estimating the position and orientation of the cornea) and iris texture decomposition (extracting features unique to an individual\u2019s iris) during training. Finally, radial texture regularization loss (a machine-learning technique that simulates smoother textures than the source material) helped further isolate and enhance the reflected scenery.<\/p>\n

Despite the progress and clever workarounds, significant barriers remain. \u201cOur current real-world results are from a \u2018laboratory setup,\u2019 such as a zoom-in capture of a person\u2019s face, area lights to illuminate the scene, and deliberate person\u2019s movement,\u201d the authors wrote. \u201cWe believe more unconstrained settings remain challenging (e.g., video conferencing with natural head movement) due to lower sensor resolution, dynamic range, and motion blur.\u201d Additionally, the team notes that its universal assumptions about iris texture may be too simplistic to apply broadly, especially when eyes typically rotate more widely than in this kind of controlled setting.\u00a0<\/p>\n

Still, the team sees their progress as a milestone that can spur future breakthroughs. \u201cWith this work, we hope to inspire future explorations that leverage unexpected, accidental visual signals to reveal information about the world around us, broadening the horizons of 3D scene reconstruction.\u201d Although more mature versions of this work could spawn some creepy and unwanted privacy intrusions, at least you can rest easy knowing that today\u2019s version can only vaguely make out a Kirby doll even under the most ideal of conditions.<\/p>\n<\/div>\n

Source<\/a><\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"

Researchers at the University of Maryland have turned eye reflections into (somewhat discernible) 3D scenes. The work builds on Neural Radiance Fields (NeRF), an AI technology that can reconstruct environments from 2D photos. Although the eye-reflection approach has a long way to go before it spawns any practical applications, the study (first reported by Tech […]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":12,"featured_media":0,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3379],"tags":[],"yoast_head":"\nResearchers reconstruct 3D environments from eye reflections • smartMILE<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/smartmileco.com\/researchers-reconstruct-3d-environments-from-eye-reflections\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Researchers reconstruct 3D environments from eye reflections\" \/>\n<meta property=\"og:description\" content=\"Researchers at the University of Maryland have turned eye reflections into (somewhat discernible) 3D scenes. The work builds on Neural Radiance Fields (NeRF), an AI technology that can reconstruct environments from 2D photos. Although the eye-reflection approach has a long way to go before it spawns any practical applications, the study (first reported by Tech […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/smartmileco.com\/researchers-reconstruct-3d-environments-from-eye-reflections\/\" \/>\n<meta property=\"og:site_name\" content=\"smartMILE\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/Smartmile-102682521688201\" \/>\n<meta property=\"article:published_time\" content=\"2023-06-28T21:09:10+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/mysterio.yahoo.com\/api\/res\/1.2\/MXWubGPmJLoEenjPlveYpg--\/ZHByPTI7dz04NzU7YXBwaWQ9ZW5nYWRnZXQ-\/https:\/\/s.yimg.com\/os\/creatr-uploaded-images\/2023-06\/0f92c550-15f2-11ee-b37b-41001507bd4d.cf.webp\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:image\" content=\"https:\/\/mysterio.yahoo.com\/api\/res\/1.2\/MXWubGPmJLoEenjPlveYpg--\/ZHByPTI7dz04NzU7YXBwaWQ9ZW5nYWRnZXQ-\/https:\/\/s.yimg.com\/os\/creatr-uploaded-images\/2023-06\/0f92c550-15f2-11ee-b37b-41001507bd4d.cf.webp\" \/>\n<meta name=\"twitter:creator\" content=\"@smartmile\" \/>\n<meta name=\"twitter:site\" content=\"@smartmile\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Organization\",\"@id\":\"https:\/\/smartmileco.com\/#organization\",\"name\":\"smartMILE & Co.\",\"url\":\"https:\/\/smartmileco.com\/\",\"sameAs\":[\"https:\/\/www.facebook.com\/Smartmile-102682521688201\",\"https:\/\/www.instagram.com\/smartmileco\/\",\"https:\/\/www.linkedin.com\/company\/smartmile-en\",\"https:\/\/www.youtube.com\/channel\/UCNDXRNG-EFmNHKgqW8lRfhg\",\"https:\/\/www.pinterest.ch\/smartMILE_Co\",\"https:\/\/twitter.com\/smartmile\"],\"logo\":{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/smartmileco.com\/#logo\",\"inLanguage\":\"en-US\",\"url\":\"https:\/\/woocommerce-331985-2347979.cloudwaysapps.com\/file\/2021\/12\/fond-blanc-rectangle.png\",\"contentUrl\":\"https:\/\/woocommerce-331985-2347979.cloudwaysapps.com\/file\/2021\/12\/fond-blanc-rectangle.png\",\"width\":2530,\"height\":454,\"caption\":\"smartMILE & Co.\"},\"image\":{\"@id\":\"https:\/\/smartmileco.com\/#logo\"}},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/smartmileco.com\/#website\",\"url\":\"https:\/\/smartmileco.com\/\",\"name\":\"smartMILE\",\"description\":\"Industries expert-approved software, services and resources\\u200b\",\"publisher\":{\"@id\":\"https:\/\/smartmileco.com\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/smartmileco.com\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/smartmileco.com\/researchers-reconstruct-3d-environments-from-eye-reflections\/#primaryimage\",\"inLanguage\":\"en-US\",\"url\":\"https:\/\/mysterio.yahoo.com\/api\/res\/1.2\/MXWubGPmJLoEenjPlveYpg--\/ZHByPTI7dz04NzU7YXBwaWQ9ZW5nYWRnZXQ-\/https:\/\/s.yimg.com\/os\/creatr-uploaded-images\/2023-06\/0f92c550-15f2-11ee-b37b-41001507bd4d.cf.webp\",\"contentUrl\":\"https:\/\/mysterio.yahoo.com\/api\/res\/1.2\/MXWubGPmJLoEenjPlveYpg--\/ZHByPTI7dz04NzU7YXBwaWQ9ZW5nYWRnZXQ-\/https:\/\/s.yimg.com\/os\/creatr-uploaded-images\/2023-06\/0f92c550-15f2-11ee-b37b-41001507bd4d.cf.webp\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/smartmileco.com\/researchers-reconstruct-3d-environments-from-eye-reflections\/#webpage\",\"url\":\"https:\/\/smartmileco.com\/researchers-reconstruct-3d-environments-from-eye-reflections\/\",\"name\":\"Researchers reconstruct 3D environments from eye reflections • smartMILE\",\"isPartOf\":{\"@id\":\"https:\/\/smartmileco.com\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/smartmileco.com\/researchers-reconstruct-3d-environments-from-eye-reflections\/#primaryimage\"},\"datePublished\":\"2023-06-28T21:09:10+00:00\",\"dateModified\":\"2023-06-28T21:09:10+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/smartmileco.com\/researchers-reconstruct-3d-environments-from-eye-reflections\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/smartmileco.com\/researchers-reconstruct-3d-environments-from-eye-reflections\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/smartmileco.com\/researchers-reconstruct-3d-environments-from-eye-reflections\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/smartmileco.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Researchers reconstruct 3D environments from eye reflections\"}]},{\"@type\":\"Article\",\"@id\":\"https:\/\/smartmileco.com\/researchers-reconstruct-3d-environments-from-eye-reflections\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/smartmileco.com\/researchers-reconstruct-3d-environments-from-eye-reflections\/#webpage\"},\"author\":{\"@id\":\"https:\/\/smartmileco.com\/#\/schema\/person\/6de69b9112eba69e1d835e383282cd40\"},\"headline\":\"Researchers reconstruct 3D environments from eye reflections\",\"datePublished\":\"2023-06-28T21:09:10+00:00\",\"dateModified\":\"2023-06-28T21:09:10+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/smartmileco.com\/researchers-reconstruct-3d-environments-from-eye-reflections\/#webpage\"},\"wordCount\":518,\"publisher\":{\"@id\":\"https:\/\/smartmileco.com\/#organization\"},\"image\":{\"@id\":\"https:\/\/smartmileco.com\/researchers-reconstruct-3d-environments-from-eye-reflections\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/mysterio.yahoo.com\/api\/res\/1.2\/MXWubGPmJLoEenjPlveYpg--\/ZHByPTI7dz04NzU7YXBwaWQ9ZW5nYWRnZXQ-\/https:\/\/s.yimg.com\/os\/creatr-uploaded-images\/2023-06\/0f92c550-15f2-11ee-b37b-41001507bd4d.cf.webp\",\"articleSection\":[\"Engadget\"],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/smartmileco.com\/#\/schema\/person\/6de69b9112eba69e1d835e383282cd40\",\"name\":\"Engadget\",\"url\":\"https:\/\/smartmileco.com\/author\/engadget\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Researchers reconstruct 3D environments from eye reflections • smartMILE","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/smartmileco.com\/researchers-reconstruct-3d-environments-from-eye-reflections\/","og_locale":"en_US","og_type":"article","og_title":"Researchers reconstruct 3D environments from eye reflections","og_description":"Researchers at the University of Maryland have turned eye reflections into (somewhat discernible) 3D scenes. The work builds on Neural Radiance Fields (NeRF), an AI technology that can reconstruct environments from 2D photos. Although the eye-reflection approach has a long way to go before it spawns any practical applications, the study (first reported by Tech […]","og_url":"https:\/\/smartmileco.com\/researchers-reconstruct-3d-environments-from-eye-reflections\/","og_site_name":"smartMILE","article_publisher":"https:\/\/www.facebook.com\/Smartmile-102682521688201","article_published_time":"2023-06-28T21:09:10+00:00","og_image":[{"url":"https:\/\/mysterio.yahoo.com\/api\/res\/1.2\/MXWubGPmJLoEenjPlveYpg--\/ZHByPTI7dz04NzU7YXBwaWQ9ZW5nYWRnZXQ-\/https:\/\/s.yimg.com\/os\/creatr-uploaded-images\/2023-06\/0f92c550-15f2-11ee-b37b-41001507bd4d.cf.webp"}],"twitter_card":"summary_large_image","twitter_image":"https:\/\/mysterio.yahoo.com\/api\/res\/1.2\/MXWubGPmJLoEenjPlveYpg--\/ZHByPTI7dz04NzU7YXBwaWQ9ZW5nYWRnZXQ-\/https:\/\/s.yimg.com\/os\/creatr-uploaded-images\/2023-06\/0f92c550-15f2-11ee-b37b-41001507bd4d.cf.webp","twitter_creator":"@smartmile","twitter_site":"@smartmile","schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Organization","@id":"https:\/\/smartmileco.com\/#organization","name":"smartMILE & Co.","url":"https:\/\/smartmileco.com\/","sameAs":["https:\/\/www.facebook.com\/Smartmile-102682521688201","https:\/\/www.instagram.com\/smartmileco\/","https:\/\/www.linkedin.com\/company\/smartmile-en","https:\/\/www.youtube.com\/channel\/UCNDXRNG-EFmNHKgqW8lRfhg","https:\/\/www.pinterest.ch\/smartMILE_Co","https:\/\/twitter.com\/smartmile"],"logo":{"@type":"ImageObject","@id":"https:\/\/smartmileco.com\/#logo","inLanguage":"en-US","url":"https:\/\/woocommerce-331985-2347979.cloudwaysapps.com\/file\/2021\/12\/fond-blanc-rectangle.png","contentUrl":"https:\/\/woocommerce-331985-2347979.cloudwaysapps.com\/file\/2021\/12\/fond-blanc-rectangle.png","width":2530,"height":454,"caption":"smartMILE & Co."},"image":{"@id":"https:\/\/smartmileco.com\/#logo"}},{"@type":"WebSite","@id":"https:\/\/smartmileco.com\/#website","url":"https:\/\/smartmileco.com\/","name":"smartMILE","description":"Industries expert-approved software, services and resources\u200b","publisher":{"@id":"https:\/\/smartmileco.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/smartmileco.com\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"ImageObject","@id":"https:\/\/smartmileco.com\/researchers-reconstruct-3d-environments-from-eye-reflections\/#primaryimage","inLanguage":"en-US","url":"https:\/\/mysterio.yahoo.com\/api\/res\/1.2\/MXWubGPmJLoEenjPlveYpg--\/ZHByPTI7dz04NzU7YXBwaWQ9ZW5nYWRnZXQ-\/https:\/\/s.yimg.com\/os\/creatr-uploaded-images\/2023-06\/0f92c550-15f2-11ee-b37b-41001507bd4d.cf.webp","contentUrl":"https:\/\/mysterio.yahoo.com\/api\/res\/1.2\/MXWubGPmJLoEenjPlveYpg--\/ZHByPTI7dz04NzU7YXBwaWQ9ZW5nYWRnZXQ-\/https:\/\/s.yimg.com\/os\/creatr-uploaded-images\/2023-06\/0f92c550-15f2-11ee-b37b-41001507bd4d.cf.webp"},{"@type":"WebPage","@id":"https:\/\/smartmileco.com\/researchers-reconstruct-3d-environments-from-eye-reflections\/#webpage","url":"https:\/\/smartmileco.com\/researchers-reconstruct-3d-environments-from-eye-reflections\/","name":"Researchers reconstruct 3D environments from eye reflections • smartMILE","isPartOf":{"@id":"https:\/\/smartmileco.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/smartmileco.com\/researchers-reconstruct-3d-environments-from-eye-reflections\/#primaryimage"},"datePublished":"2023-06-28T21:09:10+00:00","dateModified":"2023-06-28T21:09:10+00:00","breadcrumb":{"@id":"https:\/\/smartmileco.com\/researchers-reconstruct-3d-environments-from-eye-reflections\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/smartmileco.com\/researchers-reconstruct-3d-environments-from-eye-reflections\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/smartmileco.com\/researchers-reconstruct-3d-environments-from-eye-reflections\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/smartmileco.com\/"},{"@type":"ListItem","position":2,"name":"Researchers reconstruct 3D environments from eye reflections"}]},{"@type":"Article","@id":"https:\/\/smartmileco.com\/researchers-reconstruct-3d-environments-from-eye-reflections\/#article","isPartOf":{"@id":"https:\/\/smartmileco.com\/researchers-reconstruct-3d-environments-from-eye-reflections\/#webpage"},"author":{"@id":"https:\/\/smartmileco.com\/#\/schema\/person\/6de69b9112eba69e1d835e383282cd40"},"headline":"Researchers reconstruct 3D environments from eye reflections","datePublished":"2023-06-28T21:09:10+00:00","dateModified":"2023-06-28T21:09:10+00:00","mainEntityOfPage":{"@id":"https:\/\/smartmileco.com\/researchers-reconstruct-3d-environments-from-eye-reflections\/#webpage"},"wordCount":518,"publisher":{"@id":"https:\/\/smartmileco.com\/#organization"},"image":{"@id":"https:\/\/smartmileco.com\/researchers-reconstruct-3d-environments-from-eye-reflections\/#primaryimage"},"thumbnailUrl":"https:\/\/mysterio.yahoo.com\/api\/res\/1.2\/MXWubGPmJLoEenjPlveYpg--\/ZHByPTI7dz04NzU7YXBwaWQ9ZW5nYWRnZXQ-\/https:\/\/s.yimg.com\/os\/creatr-uploaded-images\/2023-06\/0f92c550-15f2-11ee-b37b-41001507bd4d.cf.webp","articleSection":["Engadget"],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/smartmileco.com\/#\/schema\/person\/6de69b9112eba69e1d835e383282cd40","name":"Engadget","url":"https:\/\/smartmileco.com\/author\/engadget\/"}]}},"fifu_image_url":"https:\/\/mysterio.yahoo.com\/api\/res\/1.2\/MXWubGPmJLoEenjPlveYpg--\/ZHByPTI7dz04NzU7YXBwaWQ9ZW5nYWRnZXQ-\/https:\/\/s.yimg.com\/os\/creatr-uploaded-images\/2023-06\/0f92c550-15f2-11ee-b37b-41001507bd4d.cf.webp","gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/smartmileco.com\/wp-json\/wp\/v2\/posts\/72615"}],"collection":[{"href":"https:\/\/smartmileco.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/smartmileco.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/smartmileco.com\/wp-json\/wp\/v2\/users\/12"}],"replies":[{"embeddable":true,"href":"https:\/\/smartmileco.com\/wp-json\/wp\/v2\/comments?post=72615"}],"version-history":[{"count":0,"href":"https:\/\/smartmileco.com\/wp-json\/wp\/v2\/posts\/72615\/revisions"}],"wp:attachment":[{"href":"https:\/\/smartmileco.com\/wp-json\/wp\/v2\/media?parent=72615"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/smartmileco.com\/wp-json\/wp\/v2\/categories?post=72615"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/smartmileco.com\/wp-json\/wp\/v2\/tags?post=72615"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}