Gérard G. Medioni

Gérard G. Medioni is a computer scientist, author, academic and inventor. He is a vice president and distinguished scientist at Amazon and serves as emeritus professor of Computer Science at the University of Southern California.^[1]

Gérard G. Medioni
Education	Ingénieur Informatique MSc., Computer Science Ph.D., Computer Science
Alma mater	Ecole Nationale Supérieure des Telecommunications (ENST) University of Southern California
Occupation(s)	Computer scientist, author, academic and inventor
Organization(s)	University of Southern California Amazon

Medioni has made contributions to computer vision, in particular 3D sensing, surface reconstruction, and object modelling. He has translated his computer vision research into customer-facing inventions and products. He has authored four books, including Emerging Topics in Computer Vision, Multimedia Systems: Algorithms, Standards, and Industry Practices, and A Computational Framework for Segmentation and Grouping, and has published more than 80 journal papers, 200 conference papers, with over 34,000 citations and his h-index is 88.^[2] In addition, he holds 103 patents to his name which include Visual tracking in video images in unconstrained environments by exploiting on-the-fly context using supporters and distracters^[3] and Depth mapping based on pattern matching and stereoscopic information, along with patents on Just Walk Out technology and Amazon One.^[4]

Medioni is a Fellow of the Association for the Advancement of Artificial Intelligence,^[5] the Institute of Electrical and Electronics Engineers,^[6] the International Association for Pattern Recognition,^[7] and the National Academy of Inventors.^[8] He is also a member of National Academy of Engineering.^[9]

Education and early career

Medioni obtained his Diplôme d'Ingénieur in 1977 from Ecole Nationale Supérieure des Telecommunications (ENST) Paris and was appointed as a Research Engineer at Thomson-CSF from 1977 to 1978. He then completed his MSc in 1980 and his Ph.D. in 1983 in computer science from the University of Southern California.^[10]

Career

Following his Ph.D., in 1983, Medioni began his academic career as a research associate professor in the Department of Computer Science and Electrical Engineering at the University of Southern California. He was subsequently promoted, becoming an assistant professor in 1987, an associate professor in 1992, and a full professor in 1999. Since 2019, he has been serving as an emeritus professor in the department of Computer Science at the University of Southern California.^[11]

From 2001 to 2007, Medioni chaired the department of Computer Science at the University of Southern California.^[1]

Medioni was the President and CEO at I.C. Vision, Chief Technical Officer at Geometrix, and Director of Research at Amazon. Additionally, he has served as an advisory board member at DXO Labs and PrimeSense in Tel Aviv. In 2019, he was promoted to Distinguished Scientist and Vice President at Amazon.^[12]

Research

Medioni's research spans the field of image understanding, focusing on fundamental issues of representation, matching, and recognition. He has also been interested in designing and implementing highly reliable vision systems capable of tackling challenging tasks, even when constructed from imperfect modules. Moreover, he used an interdisciplinary approach to connect Computer Vision and Graphics to comprehend visual information processing.^[10]

Just walk out technology

Medioni introduced the Just Walk Out technology (JWO) which is a new shopping experience for customers. The data captured by a bank of cameras and other sensors in the store is processed in real-time to solve the "who took what" problem for every customer. It achieved a high level of accuracy in detecting people, keeping track of their location throughout their journey in the store,^[13] recognizing items that a customer picks up from the shelves,^[14] and producing an accurate receipt for items they end up buying.^[15]

Amazon One

Medioni developed the algorithmic components for Amazon One. This device optically captures the unique print and vein patterns of the palm and identifies a user among enrolled users.^[16]

Primesense

As an advisory board member and technical consultant, Medioni contributed to developing a low-cost 3D depth (range) sensor, PrimeSensor, used in the Microsoft Kinect. After Apple acquired PrimeSense in 2013, the sensor was integrated into the Apple iPhone X, enabling FaceID for mobile unlock.^[17]

Tensor voting

Medioni established Tensor Voting, an approach to a wide range of problems in computer vision and machine learning that is non-parametric, data-driven, local, and requires a minimal number of assumptions. The tensor voting framework provided a unified perceptual organization methodology applicable to a wide variety of problems. While the original tensor voting formulation worked with 2-D input, it was extended to 3-D (surfaces, stereo), 4-D (motion), and N-D. It is thus applicable to both Computer Vision and Machine Learning.^[18][19]

Iterative closest point

Medioni developed the Iterative Closest Point (ICP) algorithm to create a complete 3D model of a physical object from partial scans. ICP serves as a dominant method for registering partial 3-D scans of a scene, with over 5,500 citations.^[20]

Rapid avatar capture simulation

Medioni's Rapid avatar capture and simulation was the first demonstration of using commodity depth sensors to capture the 3D shape and appearance of human subjects, and then registering it and controlling it within an animation system within minutes.^[21][22]

Face modelling

Medioni has also worked on face modeling and introduced a technique for building human face models by using only two photographs.^[23] Through collaborative research efforts he proposed a 3D face modeling and recognition system^[24] and a method to produce 3D face models in laser scan quality.^[25] Moreover, he presented a method for remotely identifying non-cooperative individuals using 3D face models from a sequence of images.^[26]

Face Recognition

Medioni has also worked on face recognition technology. He proposed domain-specific data augmentation as a more accessible way to improve face recognition, achieving performance similar to systems using large datasets.^[27] Additionally, he introduced Pose-Aware Models (PAMs) for unconstrained face recognition.^[28]

Awards and honors

• 1999 – Okawa Foundation Award, Okawa Foundation
• 2003 – Fellow, Institute of Electrical and Electronics Engineers (IEEE)^[6]
• 2004 – Fellow, Association for the Advancement of Artificial Intelligence (AAAI)^[5]
• 2007 – Most Influential Paper over the Decade Award, MVA^[29]
• 2019 – PAMI Mark Everingham Prize, IEEE Trans^[30]
• 2021 – Fellow, Asia-Pacific Artificial Intelligence Association (AAIA)^[31]
• 2021 – Distinguished Leader, APSIPA Industrial
• 2022 – Fellow, National Academy of Inventors^[8]
• 2023 – Member, National Academy of Engineering (NAE)^[9]

Bibliography

Selected books

• A Computational Framework for Segmentation and Grouping (2000) ISBN 978-0080529486.
• Emerging Topics in Computer Vision (2004) ISBN 978-0131013667
• Tensor Voting: A Perceptual Organization Approach to Computer Vision and Machine Learning (2006) ISBN 978-1598291001
• Multimedia Systems: Algorithms, Standards, and Industry Practices (2009) ISBN 978-1418835941

Selected articles

• Medioni, G., & Nevatia, R. (1985). Segment-based stereo matching. Computer vision, graphics, and image processing, 31(1), 2–18.
• Huertas, A., & Medioni, G. (1986). Detection of intensity changes with subpixel accuracy using Laplacian-Gaussian masks. IEEE Transactions on Pattern Analysis and Machine Intelligence, (5), 651–664.
• Chen, Y., & Medioni, G. (1992). Object modelling by registration of multiple range images. Image and vision computing, 10(3), 145–155.
• Stein, F., & Medioni, G. (1992). Structural indexing: Efficient 3-D object recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(2), 125–145.
• Dinh, T. B., Vo, N., & Medioni, G. (2011, June). Context tracker: Exploring supporters and distracters in unconstrained environments. In CVPR 2011 (pp. 1177–1184). IEEE.
• Khan, S., Rahmani, H., Shah, S. A. A., Bennamoun, M., Medioni, G., & Dickinson, S. (2018). A guide to convolutional neural networks for computer vision (Vol. 8, No. 1, pp. 1–207). San Rafael: Morgan & Claypool Publishers.

References

1. "USC - Viterbi School of Engineering - Viterbi Faculty Directory". viterbi.usc.edu.
2. "Gerard Medioni". scholar.google.com.
3. "Visual tracking in video images in unconstrained environments by exploiting on-the-fly context using supporters and distracters".
4. "Depth mapping based on pattern matching and stereoscopic information".
5. "Current AAAI Members Who Are Fellows". AAAI.
6. "IEEE Fellows Directory". www.ieee.org.
7. "Alphabetical List of IAPR Fellows – International Association for Pattern Recognition".
8. "Fellows". NAI.
9. "Professor Gerard Guy Medioni". NAE Website.
10. "Gérard Medioni believes now is a 'golden age' for computer vision research". Amazon Science. October 15, 2021.
11. "Gerard Medioni - IEEE Xplore Author Details".
12. "Gerard Medioni, Vice President and Distinguished Scientist, AWS Applications". US About Amazon.
13. "Locally and globally locating actors by digital cameras and machine learning".
14. "Associating events with actors using digital imagery and machine learning".
15. "Just Walk Out". Bringing Just Walk Out shopping to your stores.
16. "System for biometric identification".
17. "3D body modeling from one or more depth cameras in the presence of articulated motion".
18. "Tensor voting in N dimensional spaces".
19. "Tensor voting in N dimensional spaces".
20. Chen, Yang; Medioni, Gérard (April 1, 1992). "Object modelling by registration of multiple range images". Image and Vision Computing. 10 (3): 145–155. doi:10.1016/0262-8856(92)90066-C – via ScienceDirect.
21. Feng, Andrew; Shapiro, Ari; Ruizhe, Wang; Bolas, Mark; Medioni, Gerard; Suma, Evan (July 27, 2014). "Rapid avatar capture and simulation using commodity depth sensors". ACM SIGGRAPH 2014 Talks. Association for Computing Machinery. p. 1. doi:10.1145/2614106.2614182. ISBN 9781450329606 – via ACM Digital Library.
22. "Rapid avatar capture and simulation using commodity depth sensors".
23. Chen, Q.; Medioni, G. (1998). "Building human face models from two images". 1998 IEEE Second Workshop on Multimedia Signal Processing (Cat. No.98EX175). pp. 117–122. doi:10.1109/MMSP.1998.738922. ISBN 0-7803-4919-9. S2CID 30128026.
24. Kim, Donghyun; Choi, Jongmoo; Leksut, Jatuporn Toy; Medioni, Gerard (2016). "Accurate 3D face modeling and recognition from RGB-D stream in the presence of large pose changes". 2016 IEEE International Conference on Image Processing (ICIP). pp. 3011–3015. doi:10.1109/ICIP.2016.7532912. ISBN 978-1-4673-9961-6. S2CID 19425541.
25. Matthias Hernandez; Jongmoo Choi; Gérard Medioni (27 August 2012). "Laser scan quality 3-D face modeling using a low-cost depth camera". 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO). IEEE. ISBN 978-1-4673-1068-0.
26. Medioni, Gerard; Choi, Jongmoo; Kuo, Cheng-Hao; Choudhury, Anustup; Li Zhang; Fidaleo, Douglas (2007). "Non-Cooperative Persons Identification at a Distance with 3D Face Modeling". 2007 First IEEE International Conference on Biometrics: Theory, Applications, and Systems. pp. 1–6. doi:10.1109/BTAS.2007.4401961. ISBN 978-1-4244-1596-0. S2CID 13401437.
27. Masi, Iacopo; Trần, Anh Tuấn; Hassner, Tal; Leksut, Jatuporn Toy; Medioni, Gérard (October 11, 2016). "Do We Really Need to Collect Millions of Faces for Effective Face Recognition?". In Leibe, Bastian; Matas, Jiri; Sebe, Nicu; Welling, Max (eds.). Computer Vision – ECCV 2016. Lecture Notes in Computer Science. Vol. 9909. Springer International Publishing. pp. 579–596. arXiv:1603.07057. doi:10.1007/978-3-319-46454-1_35. ISBN 978-3-319-46453-4. S2CID 14595783 – via Springer Link.
28. Masi, Iacopo; Rawls, Stephen; Medioni, Gerard; Natarajan, Prem (October 11, 2016). "Pose-Aware Face Recognition in the Wild". pp. 4838–4846 – via openaccess.thecvf.com.
29. "Most Influential Paper over the Decade Award" (PDF).
30. "Gérard MEDIONI | World AI Cannes Festival 2023". www.worldaicannes.com.
31. "Gérard Medioni named AAIA fellow". Amazon Science. July 19, 2021.