Can you learn useful information from videogames that can be applied to real life?

These two datasets are made up of images from 6 different environments. The datasets are from 3D virtual worlds, and also real-life photographs

###Please cite this study if you use the dataset In the related study we show that if you learn from the virtual data and then transfer those CNN weights, you get better results when classifying the real world photos: https://www.researchgate.net/publication/338868329_From_Simulation_to_Reality_CNN_Transfer_Learning_for_Scene_Classification

A presentation explaining what we did, and what we achieved can be viewed here: https://www.youtube.com/watch?v=_nkpzr9NW9E This is the presentation delivered at IEEE Intelligent Systems 2020 in Varna, Bulgaria.

Context

Can you transfer learn between simulation data and real data in order to improve the classification ability of said real world data? With the growing field of sim-to-real research, scientists argue that it is not only possible, but useful too.

In this challenge, we can learn environments ("Where am I?") in an image classification experiment. Two datasets are provided, one from videogames, and another from the real world.

If you want to collect your own data in Unity, I wrote this handy script that takes images from a rotating camera and also builds the PDF just like the one included with this dataset: https://github.com/jordan-bird/Unity-Image-Dataset-Collector

Content

There are two datasets, virtual images and real images. Class labels are stored in separate csv files for ease of loading (though you can simply concatenate the column if you'd like)

There are 6 classes with numerical labels:

0 - Living Room (interior)

1 - Bathroom (interior)

2 - Staircase (interior)

3 - Forest (exterior)

4 - Field (exterior)

5 - Computer Lab (interior)

Acknowledgements

In the related study we show that if you learn from the virtual data and then transfer those CNN weights, you get better results when classifying the real world photos: https://www.researchgate.net/publication/338868329_From_Simulation_to_Reality_CNN_Transfer_Learning_for_Scene_Classification

Please cite this study if you use the dataset