Name: Controlling AWS Costs With a Data Carousel
Start: 2020-07-21T16:00:00-0400
End: 2020-07-21T17:30:00-0400

For over 20 years, ESIP meetings have brought together the most innovative thinkers and leaders around Earth observation data, thus forming a community dedicated to making Earth observations more discoverable, accessible and useful to researchers, practitioners, policy makers, and the public. The theme of the meeting is Putting Data to Work: Building Public-Private Partnerships to Increase Resilience & Enhance the Socioeconomic Value of Data.

The meeting has now ended. Check out the ESIP Summer Meeting Highlights Webinar and learn how to access session materials at https://www.esipfed.org/collaboration-updates/esip-summer-meeting-2020-recap.

Back To Schedule

Controlling AWS Costs With a Data Carousel

Feedback form is now closed.

How to manage the costs associated with a 2.4 Petabyte dataset hosted on AWS? This is a question posed by the EOSDIS Large, Mission Scale Data working group. Part of the answer lies in keeping the data in low-cost Glacier storage, however unbounded data retrieval costs are incompatible with federal budget rules. We will describe and demonstrate a data carousel model where data is restored on a fixed regular schedule and research jobs are run against the data before it is again placed in cold storage. This provides a bounded, fixed cost to NASA to operate, and allows the researchers to scale their analysis as their budgets and needs permit.

View Recording
View Session Notes
View Presentations: See attached and https://doi.org/10.6084/m9.figshare.12690038.v1

Takeaways
TBD

Speakers