Partition a 10TB table Wednesday 16:20 New York
Website: www.adyen.com
I’m Reinier Haasjes and working at Adyen as a Senior system engineer / Postgres DBA. As a system engineer I’m responsible for the whole production platform. On the production platform my main focus are the database servers.
I’m working with PostgreSQL since 2014 when Adyen was running PostgreSQL 9.2.
PostgreSQL is an important part of our production technology stack. As such, we actively engage with the community to help make it better. We use 2ndQuadrant for assistance and potential bug fixes that can be given back to the community as a result.
In order to keep the platform as stable as possible, we run a stock version of PostgreSQL, without any further modifications.
I have been to pgconf.eu for the last 4 years except last year because I just got a kid. Always really enjoyed the talks and the community. This is my first year as a spreaker.
At Adyen we have a pretty big database with a fairly high TPS on that. The size of the largest table was exceeding the 10TB and we were expecting issues with this.
At the time we were running PostgreSQL 9.4 and 9.6 with partitioning just came out.
After we already tried a lot of other stuff to improve the situation partitioning this table on a running database was the next step.
We will also talk about some features of partitioning PostgreSQL is missing to make it work for us and how we worked around this.
On one side the people who are interested in our story how we deal with big data and on the other side PG-developers who are working on the partitioning to hear a “customer story” and which features we’re missing to make partitioning work for us out of the box.
We will try to make the talk also understandable for beginners but you should know what a constraint, trigger, foreign key, etc. is.
“Improved SELECT query performance due to enhanced partition elimination during query processing and execution”. This could be a features that eliminates on of our workarounds we talk about that in our talk.
AUTO PLAN TUNING USING FEEDBACK LOOP, GPU and NVME accelerates PostgreSQL & Using Prometheus and Grafana for PostgreSQL monitoring in TomTom.