What's the difference between BigQuery and BigTable?
Yesterday night i went out with some tech friends and we started to talk about the benefit to using BigQuery and BigTable
The main question was
"is there any reason why someone would use BigTable instead of BigQuery? "
but also from the web someone ask
"I need to develop an affiliate network (thus I need to track clicks and 'sales') so I'm quite confused by the difference because bigQuery seems to be just bigTable with a better API."
We all brought our own experience on BigQuery and BigTable and in the end we believe that
The difference is basically this:
BigQuery is a query Engine for datasets that don't change much, or change by appending. It's a great choice when your queries require a "table scan" or the need to look across the entire database. Think sums, averages, counts, groupings. BigQuery is what you use when you have collected a large amount of data, and need to ask questions about it.
BigTable is a database. It is designed to be the foundation for a large, scaleable application. Use BigTable when you are making any kind of app that needs to read and write data, and scale is a potential issue.
i found this flow chart that can help us to understand
click here below to see the flow chart
https://cloud.google.com/images/storage-options/flowchart.svg
image by cloud.google.com
also on the web everyone says
Please keep in mind that Bigtable is not a relational database and it does not support SQL queries or JOINs, nor does it support multi-row transactions. Also, it is not a good solution for small amounts of data. If you want an RDBMS OLTP, you might need to look at cloudSQL (mysql/ postgres) or spanner.This may help a bit in deciding between different data stores that Google cloud offers
Which is the Cost ?
this link explains very well the cost https://stackoverflow.com/a/34845073/6785908.
i'm quoting the relevant parts here below
The overall cost boils down to how often you will 'query' the data. If it's a backup and you don't replay events too often, it'll be dirt cheap. However, if you need to replay it daily once, you will start triggering the 5$/TB scanned very easily. We were surprised too how cheap inserts and storage were, but this is ofc because Google expects you to run expensive queries at some point in time on them. You'll have to design around a few things though. E.g. AFAIK streaming inserts have no guarantees of being written to the table and you have to poll frequently on tail of list to see if it was really written. Tailing can be done efficiently with time range table decorator, though (not paying for scanning whole dataset).
If you don't care about order, you can even list a table for free. No need to run a 'query' then.
let me know your thoughts and please share your experience with us by commenting down below!
Thanks
good one