In recent years the global data volume growth has been forty percent each year, the number of bytes created daily is 10 to the power of 18 and the number of servers used by big data companies ranges up to the millions. Although a lot of the data acquired might seem useless to the general population, governments, academia, and industry are learning how to use your location check-ins, your car engine temperature log and your cell phone reception quality history to make conclusions about the public mood and opinion, about traffic patterns and about environmental parameters such as air quality.
Space related research and space based observations are great contributors of big data, satellite telemetry, sensor data, observation logs and manned space mission studies all produce enormous amounts of information that can be examined again and again to understand more about our universe, our solar system, our planet and our body.
The "Big Space Data" TP will examine the world of big data with the eyes of the space community; the TP will map the data creators, the data depositories, the data managers and data consumers related to space and space based activities. The TP will then find ways to better the data flow from sensor to processor to general public. The TP should evaluate ways of using and reusing data acquired by space based and terrestrial sensors to learn as much as possible about the universe around us, the leading question should be "how do I make new conclusions on other matters from all the data available to me". The TP will need to take into consideration the technological challenges in data management as well as the legal and ethical issues that stand between protecting privacy and intellectual property and the general good.
In the past twenty years the leading question in the acquisition of knowledge and data has changed from "is the answer out there?" to "how do I find the most relevant answer out of the data haystack?". The quests for both obtaining and presenting the most accurate and relevant data are very important to space related research. As more and more private and public organizations conduct space related missions for Earth observation, space exploration and other scientific research using advanced sensors that pour trillions of bytes into data depositories there comes a need to manage the flow and dissemination of information. Leading research and development organizations, such as NASA and Google, employ whole departments dedicated to the management of knowledge and the improvement of data flow and conservation. Other organizations such as ESA have started annual space data conferences which discuss matter of data management, processing and dissemination. There is a need to better the flow of space related knowledge not only within a single organization or research discipline but globally, to all stakeholders.
Management of data requires consideration of many aspects that range from the technicality of holding and processing the data through issues of ownership, accessibility and privacy to matters of accuracy, reliability and subjective bias prevention.
Today, most of the space related data is available on the World Wide Web. However, this data and knowledge is distributed in many depositories and points of access. Although the abundance of space related databases has its advantages such as redundancy, lower susceptibility to biases and a broad reach to different communities, it also has disadvantages in matters of quality assurance, the format unification required for research and the danger of "not seeing the whole picture".
A need for a building a central data warehouse, a database of databases arises.
The primary objectives of this Team Project include:
- Map the players and stakeholders in the knowledge flow process, from data acquisition, through data processing, data storage, data-based applications and utilities to data dissemination.
- Study the benefits and hurdles and the different aspects (research, social, legal, technical) in holding a central point of access to space related data.
- Draft a roadmap and recommend strategic actions that the space community in general and data providers (e.g., agencies, universities, companies, research institutes) in particular should implement to improve the knowledge flow process.
- Realize the positive educational experience in learning how teamwork and problem solving are achieved in an international, multicultural, and multidisciplinary environment with time and resource constraints.
- Produce a high quality report with practical and actionable recommendations that will assist decision makers and influence the future space related knowledge processes.
The tasks required to achieve these objectives include, but are not limited to:
- Research, Identify and map current situation of space data to draft the current and near future space related knowledge flow.
- Point out the obstacles, hurdles and tasks in data sharing and fusion on the legal, ownership, privacy, technical and proper use aspects.
- Draft an optimal (and feasible) situation for the flow of space related knowledge and data.
- Analyze specific case studies, in space sciences research fields and in space services fields, while referring to the current and desired states.
- Define a set of parameters that could track the implementation of the strategy in short and long term.
- Document findings and produce practical and actionable recommendations that will assist decision makers and influence the future direction of space related knowledge processes.
- Produce all required SSP deliverables that will document the team’s work, conclusions and recommendations and use these deliverables to produce conference papers to the 2016 International Astronautical Congress and the 2017 ESA Conference on Big Data from Space.
- NASA Data Portal - https://data.nasa.gov/
- NASA Office of the Chief Knoledge Officer - http://km.nasa.gov/
- Proceedings of the 2016 ESA Conference on Big Data from Space - http://congrexprojects.com/2016-events/16m05/introduction
- Proceedings of the 2014 ESA Conference on Big Data from Space - https://earth.esa.int/web/guest/events/all-events/-/article/2014-conference-on-big-data-from-space-bids-14-research-technology-and-innovation-rt-i-