We pitched MoonBuddy and were selected as finalists the following spring. With 6 months to develop for presentation at JSC, we worked to deliver a voice-controlled, AR prototype that could help astronauts plot routes, avoid obstacles, assist with experiments, and provide support during emergencies.
We pitched MoonBuddy and were selected as finalists the following spring. With 6 months to develop for presentation at JSC, we worked to deliver a voice-controlled, AR prototype that could help astronauts plot routes, avoid obstacles, assist with experiments, and provide support during emergencies.
Lunar Navigation
Lunar Navigation
Terrain Mapping
Terrain Mapping
Vital Tracking
Vital Tracking
Geological
Sampling
Geological
Sampling
Our first goal was to craft a proposal for a user interface that would address the key objectives of an EVA:
Our first goal was to craft a proposal for a UI that would address the key objectives of an EVA:
— THE CHALLENGE —
— THE CHALLENGE —
How can we assist astronauts on long-term lunar expeditions?
How can we assist astronauts on long-term lunar expeditions?
A video introduction to the Moon Buddy team and proposal.
A video introduction to the Moon Buddy team.
As NASA prepares for extended lunar missions in Artemis and beyond, astronauts have an increased need for tools that can help them overcome the physical and mental challenges of performing extravehicular activities (EVAs) on the moon.
I participated in a student challenge to develop an assistive AR HUD prototype on the Microsoft HoloLens2. We tested our product with astronauts at Johnson Space Center (JSC) and published our findings in UIST ‘22. The research intends to help inform NASA’s development of Joint Augmented Reality Visual Informatics System (JARVIS).
As NASA prepares for extended lunar missions in Artemis and beyond, astronauts have an increased need for tools that can help them overcome the physical and mental challenges of performing extravehicular activities (EVAs) on the moon.
I participated in a student challenge to develop an assistive AR HUD prototype on the Microsoft HoloLens2. We tested our product with astronauts at Johnson Space Center (JSC) and published our findings in UIST ‘22. The research is intended to help inform NASA’s development of Joint Augmented Reality Visual Informatics System (JARVIS).
— THE PROPOSAL —
— THE PROPOSAL —
A Simple, Adaptive, Voice-Controlled Display
A Simple, Adaptive, Voice-Controlled Display
Thus, our proposal centered around the following principles:
Thus, our proposal centers around the following:
Show necessary information only to avoid overwhelming astronauts or obscuring their field of view (FOV).
Show necessary information only to avoid overwhelming astronauts or obscuring their field of view (FOV).
Simplicity
Simplicity
Be responsive to mission tasks and provide command inputs to be flexible and anticipate astronauts' needs.
Be responsive to mission tasks and provide command inputs to be flexible and anticipate astronauts' needs.
Adaptive UI
Adaptive UI
VUI allows for a hands-free experience with less error compared to gesture/gaze and more customization.
VUI allows for a hands-free experience with less error compared to gesture/gaze and more customization.
Voice Control
Voice Control
Design prototypes in collaboration with Anita Sun.
Design prototypes in collaboration with Anita Sun.
— THE DISCOVERY —
— THE DISCOVERY —
Different Astronauts, Different Expertise
Different Astronauts,
Different Expertise
When it came to guiding astronauts through mission objectives, we initially thought providing high-level instructions for technical tasks would be sufficient. However, a conversation with Trevor Graff, the Chief Scientist for Astromaterials Research and Exploration Science, challenged our assumptions about an astronaut's knowledge.
When it came to guiding astronauts through mission objectives, we initially thought providing high-level instructions for technical tasks would be sufficient. However, a conversation with Trevor Graff, the Chief Scientist for Astromaterials Research and Exploration Science, challenged our assumptions about an astronaut's knowledge.
The LOOK-ODD process for geological sampling.
The LOOK-ODD process for geological sampling.
Graff emphasized that not all astronauts are geology experts, so it is still valuable to offer detailed, sequenced instructions — those with experience can always bypass them or turn it off. This reinforced our commitment to creating an adaptive UI that not only responds to specific tasks but accounts for the users' level of expertise.
Graff emphasized that not all astronauts are geology experts, so it is still valuable to offer detailed, sequenced instructions — those with experience can always bypass them or turn it off. This reinforced our commitment to creating an adaptive UI that not only responds to specific tasks but accounts for the users' level of expertise.
— THE ITERATIVE CYCLE —
— THE ITERATIVE CYCLE —
Keep it Simple, Stupid
Keep it Simple, Stupid
We found aspects to simplify with every iteration to address user test feedback and scoping issues. Some highlights…
We found aspects to simplify with every iteration to address user test feedback and scoping issues. Some highlights…
Voice Interaction Flow
Voice Interaction Flow
We initially pitched a conversational VUI thinking that it could introduce more flexibility and natural response.
We initially pitched a conversational VUI thinking that it could introduce more flexibility and natural response.
However, we realized this added too much complexity and time, as astronauts follow very precise mission plans. We streamlined the VUI into a few simple commands and leaned into a modular system for suggestions.
However, we realized this added too much complexity and time, as astronauts follow very precise mission plans. We streamlined the VUI into a few simple commands and leaned into a modular system for suggestions.
We're trying to look at the moon, not so much what's above the horizon line.
We're trying to look at the moon, not so much what's above the horizon line.
One guiding insight from our research was to keep the bulk of UI elements at the top of the astronaut's FOV, as the most important things to focus on toward the ground.
One guiding insight from our research was to keep the bulk of UI elements at the top of the astronaut's FOV, as the most important things to focus on toward the ground.
Designing for AR HUD
Designing for AR HUD
Another challenge was keeping the display uncluttered, as well as determining which elements needed to be on screen the whole time.
Another challenge was keeping the display uncluttered, as well as determining which elements needed to be on screen the whole time.
Navigational Method
Navigational Method
A subject of debate was how to display navigation information. We started with a mini-map for tracking objectives and crewmates, but it took up a lot of space and wasn't always useful.
A subject of debate was how to display navigation information. We started with a mini-map for tracking objectives and crewmates, but it took up a lot of space and wasn't always useful.
We opted for an overhead compass slightly above eye level to provide essential navigation information and directional awareness without obstructing vision.
We opted for an overhead compass slightly above eye level to provide essential navigation information and directional awareness without obstructing vision.
— FINAL DESIGN —
— FINAL DESIGN —
MoonBuddy, Start Mission
MoonBuddy, Start Mission
Keep Track of Mission Tasks
Keep Track of Mission Tasks
The tasklist provides instructions for the current task and voice command suggestions for each step.
The tasklist provides instructions for the current task and voice command suggestions for each step.
Monitor Vitals & Environmental Conditions
Monitor Vitals & Environmental Conditions
Easily check remaining time on consumables and see warnings when vitals are out of safe range.
Easily check remaining time on consumables and be warned when vitals are out of safe range.
Long-Range Navigation Assistance
Long-Range Navigation Assistance
A linear compass displays heading direction to stations, and an overhead map indicates important positions.
A linear compass displays heading direction to stations, and an overhead map indicates important positions.
Short-range Navigation Assistance
Short-range Navigation Assistance
A terrain scanner increases awareness of hidden hazards by revealing protrusions and drops (thank you, dev team 💖).
A terrain scanner increases awareness of hidden hazards by revealing protrusions and drops (thank you, dev team 💖).
Guided Steps through Science Sampling
Guided Steps through Science Sampling
Step-by-step instructions through the geological sampling process. Capture documentation hands-free.
Step-by-step instructions through the geological sampling process. Capture documentation hands-free.
— THE ROCKYARD —
— THE ROCKYARD —
Testing on the Moon
Testing on the Moon
Moon Buddy was finally ready for launch on May 23, 2022. We conducted two tests at the JSC Rockyard, a multi-acre simulation of the moon's craters, rock fields, and slopes.
Pilot Test — Houston, We Have a Problem…
Our first test was with our NASA mentor, who was well-versed in our project and the Rockyard. Despite MoonBuddy suffering from a lot of bugs, he blazed through the mission and it was almost hard to keep up. His feedback focused on improving wording and emphasizing keywords in interaction.
Final Test — Mission Success!
In our official test with a technical lead for the JARVIS program, our revisions to MoonBuddy and its presentation made the experience much smoother. It demonstrated how even someone with limited geological experience could rely on MoonBuddy to complete tasks successfully. Our tester found the design to be intuitive, easy to reference, and sufficiently minimal.
Moon Buddy was finally ready for launch on May 23, 2022. We conducted two tests at the JSC Rockyard, a multi-acre simulation of the moon's craters, rock fields, and slopes.
Pilot Test — Houston, We Have a Problem…
Our first test was with our NASA mentor, who was well-versed in our project and the Rockyard. Despite MoonBuddy suffering from a lot of bugs, he blazed through the mission and it was almost hard to keep up. His feedback focused on improving wording and emphasizing keywords in interaction.
Final Test — Mission Success!
In our official test with a technical lead for the JARVIS program, our revisions to MoonBuddy and its presentation made the experience much smoother. It demonstrated how even someone with limited geological experience could rely on MoonBuddy to complete tasks successfully. Our tester found the design to be intuitive, easy to reference, and sufficiently minimal.
Our NASA mentor taking geological samples with assistance from MoonBuddy.
Our NASA mentor taking geological samples with MoonBuddy.
HUD view from our pilot test prototype of MoonBuddy.
HUD view from our pilot test prototype of MoonBuddy.
Functionality of the prototype was a lot more important for our technical demo, so our prototype was a very stripped-down version of the final design. However, the layout, content, and interaction methods were mostly the same.
These two tests largely reflected our "different astronauts, different expertise" insight from the beginning. We were happy to see that MoonBuddy was able to assist both astronauts without hindering them from moving at the pace that they were comfortable with.
Functionality of the prototype was a lot more important for our technical demo, so our prototype was a very stripped down version of the final design. However, the layout, content, and interaction methods were mostly the same.
These two tests largely reflected our "different astronauts, different expertise" insight from the beginning. We were happy to see that MoonBuddy was able to provide assistance to both astronauts without hindering them from moving at the pace that they were comfortable with.
— FUTURE WORK —
— FUTURE WORK —
Taking Advantage of AR
Taking Advantage of AR
If we had time to develop Moon Buddy further, I would want to hone in on AR’s unique ability to directly interact with the world around you. Having AR landmarks superimposed onto the lunar surface could help give context and depth to the monotonous environment of the moon. This would be particularly useful for navigation and geological sampling.
If we had time to develop Moon Buddy further, I would want to hone in on AR’s unique ability to directly interact with the world around you. Having AR landmarks superimposed onto the lunar surface could help give context and depth to the monotonous environment of the moon. This would be particularly useful for navigation and geological sampling.
— REFLECTIONS —
— REFLECTIONS —
To the Moon and Beyond…
To the Moon and Beyond…
MoonBuddy has been the longest project I've worked on thus far, and I'm grateful for all the stellar experiences I was able to have while completing this project. From its conception to testing, some things I learned are:
🚧 Don't let ambiguity become a roadblock.
Many crucial details remained unanswered until uncomfortably close to test week. While frustrating, it was important to move forward with informed guesses and adapt to new information, rather than postponing decisions.
📚 How to craft actionable research summaries.
Approaching things in bite-sized chunks, staying in sync with development, and tailoring key insights for them was vital for effective collaboration.
💥Remain calm! Failed tests are just as meaningful.
Our first test at the rockyard was a chaotic disaster! Having made many changes just the night before, we were noticeably anxious about whether MoonBuddy would pull through. But, the most important thing is remaining calm and understanding how to improve from each test.
MoonBuddy has been the longest project I've worked on thus far, and I'm grateful for all the stellar experiences I was able to have while completing this project. From its conception to testing, some things I learned are:
🚧 Don't let ambiguity become a roadblock.
Many crucial details remained unanswered until uncomfortably close to test week. While frustrating, it was important to move forward with informed guesses and adapt to new information, rather than postponing decisions.
📚 How to craft actionable research summaries.
Approaching things in bite-sized chunks, staying in sync with development, and tailoring key insights for them was vital for effective collaboration.
💥Remain calm! Failed tests are just as meaningful.
Our first test at the rockyard was a chaotic disaster! Having made many changes just the night before, we were noticeably anxious about whether MoonBuddy would pull through. But, the most important thing is remaining calm and understanding how to improve from each test.
Angie, Matthew, and me, behind the scenes at JSC.
Angie, Matthew, and me, behind the scenes at JSC.
— SELECTED MEDIA —
— SELECTED MEDIA —
MoonBuddy in the News!
MoonBuddy in the News!
Helping Design the Next Generation of Spacesuits
engineering.com, Apr 2022
Helping Design the Next Generation of Spacesuits
engineering.com, Apr 2022
MoonBuddy Team Working with NASA on Spacesuits for Lunar Exploration
CMU News, Apr 2022
MoonBuddy Team Working with NASA on Spacesuits for Lunar Exploration
CMU News, Apr 2022
Moon Buddy, Mission Control, and Math
The Link Magazine, Issue 17.1
Moon Buddy, Mission Control, and Math
The Link Magazine, Issue 17.1
ROLE
Research & Design
Visual Identity
ROLE
Research/Design
Visual Identity
FOR
2022 NASA SUITS Student Challenge
TEAM
Anita Sun (Co-Designer)
CMU Moon Buddy Team
TIMELINE
1 year
Oct 2021 - Nov 2022
MoonBuddy ✨ is an AR heads-up display to help astronauts navigate the lunar surface and keep track of mission goals.
MoonBuddy ✨ is an AR heads-up display to help astronauts navigate the lunar surface and keep track of mission goals.
ROLE
Research & Design
Visual Identity
FOR
2022 NASA SUITS Student Challenge
TEAM
Anita Sun (Co-Design)
Moon Buddy Team
TIMELINE
1 year
Oct 2021 - Nov 2022
Our preliminary research of past Apollo transcripts, existing AR space interfaces, and interviews with former astronauts and current staff provided some guidance on understanding the challenges of the EVA journey as a whole:
Our preliminary research of past Apollo transcripts, existing AR space interfaces, and interviews with former astronauts and current staff provided some guidance on understanding the challenges of the EVA journey as a whole:
— PROJECT KICKOFF —
— PROJECT KICKOFF —
Understanding the EVA process
Understanding the EVA process
"In the spacesuit, moving your finger was like picking up a gallon of milk."
"In the spacesuit, moving your finger was like picking up a gallon of milk."
Jay Apt, Former NASA Astronaut
Jay Apt, Former NASA Astronaut
"Foreground features just blended into the background from lack of color…"
"Foreground features just blended into the background from lack of color…"
Gene Cernan, Apollo 17
Gene Cernan, Apollo 17
"Every minute on the surface with Apollo was worth a million dollars."
"Every minute on the surface with Apollo was worth a million dollars."
Ben Feist, NASA Data Visualization
Ben Feist, NASA Data Visualization
"Assisting systems [using] memorized commands can encumber the crew…"
"Assisting systems [using] memorized commands can encumber the crew…"
Dean Eppler, NASA Geologist
Dean Eppler, NASA Geologist
That we then used to form the following insights:
That we then used to form the following insights:
👨🚀
👨🚀
Spacesuits severely restrict mobility, making even simple tasks challenging.
Spacesuits severely restrict mobility, making even simple tasks challenging.
👓
👓
Navigation issues arise from reduced visibility & depth perception
Navigation issues arise from reduced visibility & depth perception
📈
📈
Prioritize gathering as much data as possible over analyzing artifacts.
Prioritize gathering as much data as possible over analyzing artifacts.
🧠
🧠
Assisting UI should integrate into existing terminology and workflow.
Assisting UI should integrate into existing terminology and workflow.
Jump to Solution→
Jump to Solution→
©2023
©2023