Skip to content
Open-SourceSprint

imagination-to-real

🌍 Mission

Our goal is to empower developers to create robots that can learn dynamic tasks like never before, enabling a world where robotics isn’t just for the elite but for everyone with an idea and the will to innovate.

🚀 What It Does

Imagination-to-Real transforms the way robots learn by providing them with realistic, diverse, and geometrically accurate visual data. The integration of Generative AI with classical physics simulators helps create synthetic datasets that are rich in variation, enabling robots to train on tasks previously deemed too complex or expensive to model.

Key Features:

  • Generative Model Integration: Combines generative models with simulators to create diverse, synthetic datasets tailored to robot training needs.
  • Temporal Consistency: Uses tools like Dreams In Motion (DIM) to ensure that the data exhibits smooth and natural transitions over time.
  • MuJoCo Compatibility: Seamlessly integrates with MuJoCo environments to facilitate data preparation for reinforcement learning and other robotic simulations.

🔑 Why It Matters

The importance of Imagination-to-Real lies in its ability to break down the barriers to entry in robotics. By providing developers with realistic synthetic data, we are eliminating the need for expensive hardware, complex sensor setups, and time-consuming data collection. This means that more people can now access the tools needed to train robots for real-world tasks, driving innovation across industries.

Not only does this make robotics more affordable and accessible, but it also opens the door to new applications in fields such as search and rescue, healthcare, automated delivery, and education — where robots can perform complex, dynamic tasks without relying on costly sensors or predefined datasets.

🏗️ Contributing to the Project

We believe in the power of collaboration, and we encourage developers, researchers, and enthusiasts to contribute to Imagination-to-Real. Here’s how you can help improve this groundbreaking project:

How to Contribute:

  • Offer New Features: If you have an idea for a new feature or improvement, we’d love to hear from you.
  • Fix Bugs: Help us make this library even better by identifying and fixing bugs.
  • Experiment with Libraries: Try out the existing libraries in creative ways and submit your findings or experiments.

You can contribute to the project by submitting a pull request on our GitHub repository.

Here’s the link to get started: GitHub Repository

If you’re not sure where to start, feel free to raise an issue for any bugs or feature requests, and we’ll work together to solve them.

🎉 What You Will Get

By contributing to Imagination-to-Real, you open doors to exciting opportunities:

  • Collaboration Opportunities: Work with like-minded innovators from around the globe to push the boundaries of robotics and AI.
  • Access to Resources and Datasets: Utilize top-notch tools and datasets to bring your ideas to life.
  • Guidance from Experts: Gain valuable insights and mentorship from experienced professionals in the field.
  • Recognition and Rewards:
    • Get listed on the Contributor Wall for your contributions.
    • Job opportunities if your collaboration is selected for further development.
    • The SmilingRobo Tag to showcase on your resume and demonstrate your involvement in cutting-edge open-source projects.
    • Shoutouts on SmilingRobo’s social media platforms, recognizing your contributions to the community.

We thank the authors of LucidSim for their opensource code and Extreme Parkour for their open-source codebase, which we used as a starting point for our library.

Citation

If you find our work useful, please consider citing:

@inproceedings{yu2024learning,
  title={Learning Visual Parkour from Generated Images},
  author={Alan Yu and Ge Yang and Ran Choi and Yajvan Ravan and John Leonard and Phillip Isola},
  booktitle={8th Annual Conference on Robot Learning},
  year={2024},
}

Github imagination-to-real