Data Storage Options

The full suite of storage service options, varying on your application and workload are
(Check resume graphic at 2:20)

Product Simple description Ideal for
Cloud Storage binary/object store large or rarely accessed unstructured data
Datastore NoSQL. Scalable store for structured serve structured pure-serve use cases
Firestore NoSQL. Cloud-native app data at global scale real time NoSQL database to store and sync data
BigTable NoSQL. High volume low latency database heavy read/write or analytical data
CloudSQL SQL. VM RDBMS Web frameworks, existing apps
Spanner SQL. Relational DB system low latency transactional systems
BigQuery Auto-scaling analytic data warehouse interactive analysis of static datasets

Google Cloud main storages

Cloud Storage

Unified object storage. Allows to serve, analyze and archive data.

  • Fully managed, highly reliable
  • Cost efficient, scalable blob store
  • Objects access via HTTP requests, including ranged gets to retrieve data portions
  • Object name is the only key
  • Objects are treated as bytes with no structure

Ideal for:

  • Images and videos
  • Blobs
  • Unstructured data
  • Static website hosting

Cloud Datastore

Fully managed, auto scalable NoSQL database. It automatically handles sharding and replications. Provides SQL-like queries. In the future they will be migrated to Cloud Firestore.

Ideal for:

  • semi-structured data
  • durable key-value data
  • managing multiple indexes
  • transactions

Use cases include product catalogs, and user profiles.

Cloud Firestore

Is the next major version of Cloud Datastore. Is a fully managed, scalable, serverless NoSQL database. Secutiry access controls for data are built in and enable you to handle data validation via a config language.

It introduces new features such as

  • new strongly consistent storage layer
  • collection and document data model
  • Native mobile and web client libraries
  • Real-time updates

Ideal for:

  • document-oriented data
  • large collections of small documents
  • native mobile and web clients
  • durable key-value data
  • hierarchical data
  • managing multiple indexes

It is backwards compatible with Cloud Datastore but the new features aren’t. To access all the new features, you have to use it in native mode.

Cloud Bigtable

High performance NoSQL database. Sparselely populated table that can scale to billions of rows and thousands of columns. It can store TB to PB of data.

It’s built for fast key value lookup and scanning over a defined key range
Updates to individual rows are atomic
It offers sub-ten ms latency
Changes to deployment config are immediate. No downtime. It supports Hbase API

Ideal for:

  • Opeartional applications
  • Analytical applications
  • Storing large amounts of single key-value data
  • MapReduce operations

Cloud SQL

This is GCP’s relational database service. It automanages replication, failover and backups. It supports MySQL and PostgreSQL. You can replicate a master instance to one or more read-only replicas.

Is it possible to configure and use it through a Google proxy.

Ideal for:

  • Web frameworks
  • Structured data

Cloud Spanner

Fully managed relational database service. Offers strong consistency and horizontal scalability. Provides automatic synchronous replication.
It’s built for multi-region, mission-critical applications.

It differs with Cloud SQL in that Cloud Spanner uses real primary keys and has the notion of interleave child records instead of foreign keys.

Considerations to take when working with Cloud Spanner:

  1. Avoid creating hotspots in your database when choosing a primary key.
    Those are created by choosing a column whose value monotonically increases the first part of the key. This is because it results in all inserts occurring at the end of your key space. Cloud Spanner divides data among servers by key ranges, so all your inserts will be directed at a single server. A solution is to hash the keys.

  2. An interleave table is a table that you declare to be a child of another table, because you want the rows of the child table to be physically stored together. The predix of the primary key of a child table must be the primary key of the parent table

  3. When loading data make sure that it writes from multiple workers.

  4. Avoid creating non-interleaved indexes on columns with monotonically increasing or decreasing keys.

  5. Create indexes after you bulk load your data.

BigQuery

Low cost enterprise data warehouse for analytics.

  • fully managed
  • PB scale
  • serverless
  • fast response times

Ideal for:

  • OLAP (Online Analytic Processing) workloads
  • Big data exploration and processing
  • Reporting via Business Intelligence (BI) tools

Other storages information

Microsoft SQL Server images

You don’t need to use Cloud SQL to run a SQL server. They come automatically pre-installed at VMs with licensing from Microsoft but they are not a managed service like Cloud SQL or Cloud Spanner.

Firebase Storage

Firebase is a mobile and web app development platform.
Check at 1:50 table

Cloud Storage for Firebase

Stores user generated data in files to Google Cloud Storage. It’s SDK provides easy authentication methods.

Ideal for:

  • user generated iamges, pictures and videos
  • blobs

Firebase Realtime Database

It also has a real time database and it allows you to store and sync data with Firebase’s NoSQL Cloud Database. Data is synced across all clients in real time. Ideal for:

  • mobile and web apps which require responsiveness when offline

Firebase Hosting

Fast and secure way to host static resources for your web app.

Ideal for:

  • URL rewriting
  • Atomic release management
  • Firebase integration

Cache app data

You can use Redis Labs Memcached Cloud to cache app data on GCP.

  • fully managed
  • 30 MB of cache at no charge