Oker, A., Prigent, E., M. Courgeon, Eyharabide, V., Urbach, M., Bazin, N., Amorim M.A., Passerieux, C., Martin, J.C., Brunet-Gouet, E. (2015) How and why affective and reactive virtual agents will bring new insights on social cognitive disorders in schizophrenia? An illustration with a virtual card game paradigm., in: Frontier Human Neuroscience vol 9 (133) (i.f. 3.0) 452 downloads
Oker, A., M. Courgeon, Prigent, E., Eyharabide, V., Bazin, N., Urbach, M., Passerieux, C., Martin, J.C., Amorim, M.A. & Brunet-Gouet, E. (2015) Rating an affective agent's help: a virtual reality study of help and metacognition, in: International Journal of Synthetic Emotions , vol 6(1) pp 60-73 (i.f. 0.64)
M. Courgeon, G. Rautureau, J.C. Martin, O. Grynszpan (2014) Joint Attention Simulation using Eye-Tracking and Virtual Humans, in: IEEE Transactions on Affective Computing, vol 5 (3) pp 238 - 250 (i.f 2.675) 766 downloads
S. Buisine, M. Courgeon, A. Charles, C. Clavel, J-C. Martin, N. Tan, O. Grynszpan (2013) The role of body posture in the recognition of emotion in contextually-rich scenarios, in: Intnl. Journal of Human-Computer Interaction, juillet 2013 (i.f. 1.131) 580 downloads
M. Courgeon, C. Clavel (2013) MARC: a Framework that Features Emotion Models for Facial Animation during Human-Computer Interaction, in: Journal on Multimodal User Interfaces - Avril 2013 (i.f. 0.833)
T. Vanhala, V. Surakka, M. Courgeon, and J.C. Martin (2012) Voluntary Facial Activations Regulate Physiological Arousal and Subjective Experiences during Virtual Social Stimulation, in: ACM Transactions on Applied Perception, ACM TAP, vol. 9(1), pp 1-21 (i.f. 1.15) 759 downloads
M. Courgeon, C. Clavel, N. Tan, J.C. Martin (2011) Front View vs. Side View of Facial and Postural Expressions of Emotions in a Virtual Character, in: TOE Springer LNCS Transactions on Edutainment, vol 6(1), pp 132-143 612 downloads
 O. Grynszpan, J. Nadel, J. Constant, F. Le Barillier, N. Carbonell, J. Simonin, J-C. Martin, M. Courgeon (2011) A new virtual environment paradigm for high functioning autism intended to help attentional disengagement in a social context in: JOPTE Journal of Physical Therapy Education, vol 25(1), pp 42-47 (i.f. 3.113)
L. Rajaonarivo, M. Courgeon, E. Maisel, P. De Loor (2017) Inline Co-Evolution between Users and Information Presentation for Data Exploration, in: 22nd International Conference on Intelligent User Interfaces , Mar 2017, Limassol, Cyprus 55 downloads
M. Courgeon, C. Hoareau, D. Duhaut (2016) Interaction with Artificial Companions: Presentation of an Exploratory Study, in: 8th International Conference on Social Robotics (ICSR 2016), Nov 2016, Kansas City, MO, United States. Springer, Lecture Notes in Computer Science, 9979, pp.611-620 45 downloads
M. Courgeon, D. Duhaut (2015) Artificial Companions as Personal Coach for Children: The Interactive Drums Teacher, in: 12th Internationnal Conference on Advances in Computer entertainment Technologies - ACE 2015, 16-19 nov. 2015, Malaysie
M. Courgeon, C. Clavel, J.C. Martin (2014) Modeling Facial Signs of Appraisal During Interaction: Impact on Users' Perception and Behavior, in: International Conference on Autonomous Agents and Multi Agents Systems, pp 765-772, AAMAS 2014, Paris, France, 5-9 mai 2014 (acceptation ~ 23%) 915 downloads
T. Giraud, M. Courgeon, M. Tardieu, A. Roatis, X. Maitre (2014) A Three-Dimensional Mirror Aaugmented by Medical Imaging: Questioning Self-Portraying at the Limit of Intimacy, in: ACM CHI Conference on Human Factors in Computing Systems, pp 845-854, ACM-CHI 2014, Toronto, Canada, 28-30 avril 2014
C. Becker-Asano, P. Stahl, M. Ragni, M. Courgeon, J.-C. Martin, B. Nebel (2013) An affective virtual agent providing embodied feedback in the paired associate task: system design and evaluation, in: Thirteenth International Conference on Intelligent Virtual Agents, pp 406-415, IVA 2013, Edinburgh, UK, 29-31 août 2013 (acceptation ~18%) 513 downloads
M. E. Hoque, M. Courgeon, B. Mutlu, J-C. Martin, R. W. Picard (2013) MACH: My Automated Conversation coacH, in: ACM International Joint Conf. on Pervasive and Ubiquitous Computing, pp 697-706, UbiComp 2013, 8-12 septembre 2013, Zurich, Suisse, Best Paper Award (acceptation ~18%) 1763 downloads
M. Boukhris, M. Courgeon, C. Jover, J-C Martin, G. Gambier, C. Guiard (2013) A Conceptual Framework studying the fidelity of Expressive Digital Doubles, in: Proceedings of the 26th International Conference on Computer Animation and Social Agents, CASA 2013, 16-18 mai 2013, Istanbul, Turquie 449 downloads
M. Courgeon, O. Grynszpan, S. Buisine, J-C. Martin (2012) Interactive expressive virtual characters: challenges for conducting experimental studies about multimodal social interaction, in: International Conference Series on Disability, Virtual Reality and Associated Technologies, ICDVRAT 2012, 10-12 septembre 2012, Laval, France
M. Courgeon, C. Clavel, N. Tan, J.C. Martin (2011) Front View vs. Side View of Facial and Postural Expressions of Emotions in a Virtual Character, in: Proceedings of the 24th Conference on Computer Animation and Social Agents, CASA 2011, 26-28 mai 2011, Chengdu, Chine 612 downloads
N. Tan, G. Pruvost, M. Courgeon, C. Clavel, Y. Bellik, J.C. Martin (2011) A Location-Aware Virtual Character in a Smart Room: Effects on Performance, Presence and Adaptivity, in: Proceedings of the International Conference of Intelligent User Interface, pp 399-402, IUI 2011, 13-16 février 2011, Palo Alto, CA, USA, (acceptation ~ 30%) 454 downloads
M. Courgeon, M-A. Amorim, C. Giroux, J-C. Martin (2010) Do Users Anticipate Emotion Dynamics in Facial Expressions of a Virtual Character? in: Proceedings of the 23rd International Conference on Computer Animation and Social Agents, CASA 2010, 31 Mai-2 juin 2010, St Malo, France 506 downloads
C. Jacquemin, W.K. Chan, M. Courgeon (2010) Bateau Ivre: An Artistic Markerless Outdoor Mobile Augmented Reality Installation on a Riverboat. in: Proceedings of ACM Multimedia 2010 Interactive Art, pp 1353-1362, 25-29 octobre 2010, Florence, Italie (acceptation ~ 22%) 500 downloads
M. Courgeon, S. Buisine, J-C. Martin (2009) Impact of Expressive Wrinkles on Perception of a Virtual Character's Facial Expressions of Emotions, in: Proceedings of the 9th International Conference on Intelligent Virtual Agents, IVA 2009, Lecture Notes in Computer Science, vol. 5773, pp 201-214, Amsterdam, Pays Bas (acceptation ~ 19%) 609 downloads
M. Courgeon, C. Jacquemin, J-C. Martin (2008) User's Gestural Exploration Of Different Virtual Agents' Expressive Profiles, in: AAMAS'08 Proceedings of the 7th international joint conference on Autonomous Agents and Multi-Agent Systems, AAMAS 2008), vol 3(1), pp 1237-1240, 16-18 mai 2008, Estoril, Portugal (acceptation ~ 16 %) 515 downloads
J-C. Martin, C. Jacquemin, L. Pointal, B. Katz, C. D'Alessandro, A. Max, M. Courgeon (2007) A 3D Audio-Visual Animated Agent for Expressive Conversational Question Answering in: Proceedings of the Auditory-Visual Speech Processing Conference, AVSP 2007, 512 downloads
M. Courgeon,MARC : Modèles Informatiques des Emotions et de leurs Expressions Faciales pour l'Interaction HommeMachine Affective Temps Réel, in: PhD Thesis, University of Paris South 728 downloads
M. Courgeon,Simulation et contrôle de visages virtuels expressifs en situation d'interaction, in: Master Thesis, University of Paris South 665 downloads
C. Chastagnol, C. Clavel, M. Courgeon, L. Devillers (2014) Designing an Emotion Detection System for a Socially-Intelligent Human-Robot Interaction, in: Natural Interaction with Robots, Knowbots and Smartphones, pp 199-211
M. Courgeon, T. Giraud, X. Maitre (2015) Mirroir en réalité augmentée pour l'imagerie médicale interactive, in: Conférence francophone IHM 2015, Octobre 2015, Toulouse, France
M. Courgeon, G. Rautureau, J-C. Martin and O. Grynszpan (2013) Humains Virtuels et Eye-Tracking: Regard Partagé et Attention Conjointe, in: Intercompehension de l'Intraspecifique a l'Interspecifique, III 2013, 30sept-1oct 2013, Lorient, France 990 downloads
T. Giraud, M. Courgeon, F. Lafont, X. Maître, A. Roatis and M. Tardieu (2013) Première intimité de l'être : Un miroir tridimensionnel augmenté par imagerie médicale, in: Journées nationales du Développement, JDev 2013
L. Philip, C. Clavel, M. Courgeon, and J.C. Martin (2013) An Affective Computing Platform for Investigating the Role of Pedagogical Feedback in Computer Mediated Collaboration, in: Alpine Rendez-Vous (ARV) 2013 Workshop on Tools and Technologies for Emotion Awareness in Computer-Mediated Collaboration and Learning 406 downloads
M. Courgeon, J-C. Martin, G. Rautureau, O. Grynszpan (2013) Simulation de mécanismes d'attention conjointe à l'aide d'un eyetracker et d'un humain virtuel, in: Journée GT-ACAI 2013, 7 juin 2013, Paris, France
M. Courgeon, C. Clavel, J-C. Martin (2012) Approche Comparative des modèles informatiques des émotions pour l'animation faciale en situation d'interaction in: Workshop Affect, Compagnon Artificiel, Interaction, WACAI 2012, Grenoble, France, 15-16 novembre 2012 686 downloads
A. Clay, N. Couture, L. Nigay, J-B. de la Rivière, J-C. Martin, M. Courgeon, M. Desainte-Catherine, E. Orvain, V. Girondel, G. Domenger (2012) Interactions and Systems for Augmenting a Live Dance Performance, in:International Symposium on Mixed and Augmented Reality, ISMAR 2012, Atlanta, USA (taux d'acceptation ~ 30%) 1173 downloads
N. Tan, C. Clavel, M. Courgeon, J-C. Martin (2010) Postural Expression of Action Tendency, in: Proceedings of the ACM Multitimedia 2010 Workshop on Social Signal Processing, 25-29 octobre 2010, Florence, Italie 454 downloads
M. Courgeon, M. Rebillat, B. Katz, C. Clavel, J-C. Martin (2010) Life-Sized Audiovisual Spatial Social Scenes with Multiple Characters: MARC & SMART-I2, in: Proceedings of AFRV 2010, Workshop de l'Assocation Française de Réalité Virtuelle, 6-8 décembre 2010, Orsay, France 621 downloads
M. Courgeon, C. Clavel, J-C. Martin (2009) Appraising Emotional Events during a Real-time Interactive Game in: Proceedings of the International workshop on Affective-aware Virtual Agents and Social Robots, ICMI09, 9 novembre 2009, Boston, USA 708 downloads
A. Clay, M. Courgeon , N. Couture, E. Delord, C. Clavel, J.-C. Martin (2009) Expressive Virtual Modalities for Augmenting the Perception of Affective Movements in: Proceedings of the International workshop on Affective-aware Virtual Agents and Social Robots, ICMI09, 9 novembre 2009, Boston, USA 514 downloads
M. Courgeon, J-C. Martin, C. Jacquemin (2008) MARC: Un Personnage Virtuel Réactif Expressif, in: Proceedings of WACA 2008 the 2nd Workshop on Animated Conversational Agents, 28 novembre 2008, Paris, France 635 downloads
C. Jacquemin, M. Courgeon, J-C. Martin (2008) Expressive Facial Animation : The MARC Project at LIMSI, in: SyysGraph seminar 2008, Finland 312 downloads
M. Courgeon, J-C. Martin, C. Jacquemin (2008) MARC: a Multimodal Affective and Reactive Character, in: Proceedings of the 1st ICMI Workshop on Affective Interaction in Natural Environments, AFFINE 2008, 20-22 Octobre 2008, Chania, Grèce 658 downloads
Research Engineer résumé
Tools and Languages
- Java, C#, C, C++, python
- OpenVR, OpenCL, OpenGL
- GLSL, nVidia CG
Development Tools & IDE
- Research Project Managment
- Students Supervision
- Virtual and Augmented Reality Technologies (HMD, OpenVR, Cave, etc.)
- GPU / GP-GPU Technologies
- Distributed system and network applications
- Object Oriented Programing
How to visit unsafe locations or unavailable historical sites? Using Virtual reality, we developped a virtual immersive visit of Trevarez Castle, France.
Visitors were invited to test the application during the last weekend of october. We tried different approaches for sharing the experience between two visitors.
The project was really well received by visitors. We hope the renew the experience soon.
Working with Arnaud Uguen from Brest hospital, we designed a tool for teaching med students to manipulate 3D representation of anapathic data. The goal is to help students to acquire a 3D mental representation of these data instead of the usual 2D view.
Several exercises can be implemented. This work is still in progress, and evaluation with students is pending.
Working with Astrophysicians from the Institue for Spatial Astrophysics (Orsay), i made a GPU erosion model to simulate Mars surface evolution.
Meanwhile, I migrated EGL in C# to use OpenVR and thus the HTC Vive. This video shows a fine scale interactive simulation of erosion. The controlers can add water and interact with the fluid
Note: Near the end of this video, you'll see abrupt changes of the terrain: I simply pressed the "reset heightmap" button.
Based on the work done for the black hole installation at "La cité de sciences et de l'industrie", I use the newly "Vive compatible" EGL to create a shared environement for exploring and teaching general relativy. Two users can share the space, shot rays, and observe time distorsion using HMD virtual reality.
Part of the MOCA ANR project I was working on required a drum teacher for children. This is the first version of the drums teacher used during the final MoCA experiment with children.
As part of an exhibition on light through the universe at "La cité des sciences et de l'industrie" in Paris, we created a black-hole lens simulator. The center cylinder holds a black hole (scale 1/1200000) that deform light for the other side. A series of Kinect sensor detect user in space. The video here is a simulation of the global system made prior to the real installation. Working on a simulated space allows us to create the installation virtualy and then deploying it on site once ready.
The light paths are physicaly based on general relativity equations, and uses Schwarzschild approximation of geodesic trajectory.
I rewrote the core rendering system of my OpenGL engine. It's written on OpenGL4.4.
It's now full Deferred Rendering with global illumiation, bloom, screen space ambient occlusion and screenspace reflections,It also compute up to 6 shadow maps.
I implemented a dynamic sky lighting. I only change the time of day and global illumination is automatically adapted.
Sea waves are GPU simulated using a physical model I borrowed from colleague's research. The simulation is also done in CPU but only at the boat position (as Bullet is CPU based). The boat is a Bullet Rigid body to which buoys are attached. These buoys apply local forces when immerged in the water. The force intensity is visible on the second part of the video, and is represented as blue bars, while the buoys are green spheres.
Part of the MOCA ANR project I was working on required a drum teacher for children. This drums teacher was later used on a TV screen, but MARC was compatible with the Cave system at the European Center for Virtual Reality. Drums is played by someone outside the Cave, and based only on the strokes (Xbox Drum Kit), the agent creates a similar motion in the immersive setup. It's thus a "Wizard of Oz" setup.
Based on my 2D implementation (Compute Shader based), I extended to 3D cells. The result looks great, but the bounds and performance are a limit to apply this kind of algorithms in games.
My first time implementing Navier-Stokes equations on GPU. I started by a simple multipass Vertex/Fragment shaders implementation, then moved to Compute Shaders, which are faster, and allow for 3D voxels.
Working with ADN-DA on high quality digital doubles, I reimplemented from scratch Jorge Jimenez's Screen Space Subsurface Scatering using EGL and OpenGL 4. The algorithms are physicaly based. While we see this in games nowaday, the multi pass rendering process was not really compatible at the time with real game engine pipelines, yet the result was astonishing (in 2012) and real time.
The rendering process is composed of : - Multiple Shadow Mapping
- Separable Sub Surface Scattering for humain skin
- Depth of Field
A collaboration with Rosaling Picard's Team (intiator of the research domain called Affective Computing) brings MARC to the M.I.T MediaLab. The goal was to create the first virtual agent for job interview training.
Using latest GPU technology, we created a digital mirror allowing to view the full volumetric scan of the human body. Deforming a 3D data matrix in real-time using a kinect sensor skeleton motion was challenging.
The mirror was first part of a 8 month show at "le musée des arts et métiers" in Paris, and was shown is several other occasions. It is still visible nowaday in the PASS museum (Mons, Belgium) and at "La cité des sciences et de l'industrie", in Paris.
After integrating JPhysX into my Game library, I wanted something more complete. I switched to JBullet, which is much more complete than JPhysX. This video shows the resulting scene.
Two years after the first application, MARC became a widely used tookit among the academic research on virtual agent. Engine such as Unity 3D are not yet available in 2010, meaning that academic research must rely on a few agent animation softwares, and MARC bring a finer facial animation and better modularity than existing systems.
E.G.L. was originaly only a rendering library. In early 2010, I integrated nVidia PhysX into the engine. This demo was a first test of gravitation, magnetic forces and collision of arbitrary shapes, and chains constrains.
Unfortunatly, JPhysX (java wrapper) was not maintained. The library latter moved to Bullet Physics.
The Multimodal Affective and Reactive Character Toolkit started in 2008, as the beginning of my PhD. MARC is a virtual agent plateform, oriented toward facial animation and expression of emotion. Originaly based uppon VirChor (by C. Jacquemin), I started to develop tools to edit and control the virtual agent using E.G.L (Java). In the end, I reimplemented the whole toolkit in java.
MARC latter evolved to include full body animation, mocap integration, and virtual reality in the EVE cave system (LIMSI-CNRS).
MARC is the main software result of my PhD, and can still be downloaded here.
Resulting of a encounter with Carole Arrat, Digital Artist, Particle is a self evolving matter responding to user biological signals. It's based on GP-GPU (OpenGL 3) programming and rendering to simulate thousands of particles in real time, plugged into biological sensors and real time video stream.
Following C. Jacquemin master's courses, I started the E.G.L. library.
E.G.L. is a programmer friendly engine that allow fast prototyping of shaders, easy integration of rendering and interaction devices. It's in many way similar to engines such as BabylonJS or Ogre3D.
Originaly written in JAVA/OpenGL 2.1, it evolved with OpenGL 3 and 4 during the last decade. In 2015, EGL was ported in C#. It's now based on OpenGL 4.5 / OpenAL / OpenVR, and uses Bullet physics.
As you can guess, pretty much everthing changed since 2007, but EGL is still the engine I use in most of my work.
The video here is one of my first attempt at shader programming. A parrallax mapping Shader.
Following C. Jacquemin master's courses, I started programming shaders using C++ and nVidia CG language. Shadow mapping, Ray tracing, Environment mapping, and so on.
This is my 3rd year project, the very begining of my OpenGL skills. Inspired by Stargate Atlantis (TV Show) the program used no shader, only the OpenGL 2.1 fixed pipeline. All geometries are procedural, including the Jumper ship, the montains, and the stargate. Flypath was defined using a set a bezier curves.
In the first part, user is free to walk around and in the ship. The walk path is limited by some collision planed. Then, the fly mode is enabled, the ship fly along a fixed path. Camera can be changed in real time.