This research aims to develop a new approach for visual deep reinforcement learning that better generalizes to various visual randomization such that a trained policy can be deployed in the real world in a zero shot manner. The new approach should be robust to visual perturbations and changes. Our main goal is to do better at visual generalization benchmarks.
A key concept of the new approach is to leverage foundational semantic segmentation models to get as much information from an image as possible. Semantic segmentation will also be used to learn auxilary tasks during training that will learn important parts of an image for the task at hand.
The proposed method should prioritize easy real-world deployment (eg: requiring only RGB camera(s)) and leverage existing image encoders.
Initial efforts will focus on single task visual deep reinforcement learning, with the objective of extending the solution to goal-conditioned settings.
Below is a birds eye view of a very approximate long term planning for this research. The goal is not to plan everything in great details but rather to offer a general idea of how this research will be conducted, the details will be addressed on-the-fly and adjustments will certainly be made as the research progresses.
The board below shows a short term planning for this research, it is more granular and represents exactly what is being worked on. The board’s todo column should not extend 1-2 weeks of planning as it is hard to plan past this period. The goal of this board is to offer a concrete and easy way to organize what is being worked on for the week and it focuses on day-to-day work. It also allows re-prioritization and quick adjustments based on feedback.
Click the page below to see the planning :
Below is a link to a mind map that visually shows the different topics that are being explored as part of this research. It also includes rough notes that are note yet synthesized This page also includes brainstorming ideas and intuitions behind the explored topics and potential ideas for a new approach.
Click the page below to see the brainstorming ideas :