Lizard Global's Guide to Firestore Database Backups
Get the latest updates about our blog posts.
Subscribe so you don’t miss out!
At Lizard Global, cutting-edge software engineering runs through the veins of every single member of our development team. They are experts at turning an app idea into a well-functioning and high-performance piece of software. In order to give you a better understanding of the processes of software development, we provide you with a sneak peek behind the screens. This blog dives deeper into the use of application databases and database backups in particular, while taking a look at the following topics:
- The basics of database backups
- Why developers use database backups
- Daily Firestore backups
- Creating a bucket
- Cloud function
- Testing of Cloud scheduler and Cloud function
- Viewing export data
Database backups: what and why?
Database backups are designed to create a clone of a production database that can be recovered in the event of data failure or a potential cyber-attack. Data failure can be caused by faulty hardware, software failure, data corruption, malicious attack, human-caused error, and accidental data deletion. These database backups are essential for software developers, as it can save a lot of time and effort if things go wrong. In order to keep the development process as efficient and agile as possible, it’s important to make sure all data is stored safely and backed up.
Database backups allow data to be restored from an earlier point in time to help companies be prepared for unplanned events. It’s important to store this backup data on a separate server to protect against primary data loss or corruption. The separate server can be disk storage or cloud storage container. For the best result, a backup has to be made on a regular basis to minimize the amount of data lost in between backups. The longer the gap between backup copies, the higher the risk of data loss when recovering from backup.
Daily Firestore backups
In this blog we are going to dive deeper into how database backups actually work. We’re going to focus on daily Firestore backups, and take you on a step-by-step journey towards a routine of frequent backups.
Create a bucket
Buckets are the basic containers that hold your data. Everything that you store in Cloud Storage, must be contained in a bucket. You can use buckets to organize and manage your data, as well as controlling the access to your data. At lizard Global, we use buckets to store our database backups.
- Create a new bucket by going to the Storage section in GCP (Google Cloud Platform)
- After creating a new bucket, navigate to ‘lifecycle rules’ and click on the rule
- Add rules to delete backup objects after a specific number of days
Cloud Function
Google Cloud Functions is a serverless execution environment for building and connecting cloud services. With Cloud Functions, you can write simple, single-purpose functions that are attached to events emitted from your cloud infrastructure and services. At Lizard Global, we use Cloud Functions to manage backup scripts, which are triggered by Cloud Scheduler.
- Create a cloud function with following code:
const functions = require('firebase-functions');
const firestore = require('@google-cloud/firestore');
const client = new firestore.v1.FirestoreAdminClient();
// Replace BUCKET_NAME
const bucket = 'gs://BUCKET_NAME';
exports.scheduledFirestoreExport = functions.pubsub
.schedule('every 24 hours')
.onRun((context) => {
 const projectId =
  process.env.GCP_PROJECT || process.env.GCLOUD_PROJECT;
 const databaseName =
   client.databasePath(projectId, '(default)');
 return client.exportDocuments({
  name: databaseName,
  outputUriPrefix: bucket,
  // Leave collectionIds empty to export all collections
  // or set to a list of collection IDs to export,
  // collectionIds: ['users', 'posts']
  collectionIds: []
  })
 .then(responses => {
   const response = responses[0];
   console.log('Operation Name: '+response['name']);
 })
 .catch(err => {
   console.error(err);
   throw new Error('Export operation failed');
 });
});
- Replace these variables:
- Replace BUCKET_NAME with the name of your bucket.
- Modify collectionIds : [] to export only the specified collection groups. Leave as-is to export all collections.
- Deploy the scheduled function
Test Cloud Scheduler and Cloud Function
Cloud Scheduler is a fully managed enterprise-grade job scheduler. It allows you to schedule any job or task in a virtual environment. It is essential to test your scheduler on a regular basis to make sure it’s working properly, and backups have been generated.
- Navigate to Cloud Scheduler in GCP
- In the row for your cloud scheduler, click on Run now
- After a few seconds, the Cloud Scheduler job should update the result column to Success and Last run to the current time. You may need to click Refresh.
The Cloud Scheduler page only confirms that the job is called your Cloud Function. Be sure to open the Cloud Function page to see your function's logs.
View Export Data
After the export operation is completed, new files should be in your cloud storage bucket. And there you have the database backup that will save you a lot of hassle in the potential case of data failure.
Need a hand?
At Lizard Global, we’re working in an agile and transparent way to make sure our clients and readers are continuously up-to-date with the development status of their applications. That’s why we like to tell you all about the actual processes that take place behind the scenes of Lizard Global. Because these articles can be a bit complex, we’re always here to help and answer any questions you may have after reading our articles. Our experts gladly tell you more about their work, so you know exactly what the development process at Lizard Global looks like. We’re also open for requests, so if you have a topic you want to know more about, let us know, and we dive into it! Get in touch with us via the contact form or chatbox on our website, or send us a message on WhatsApp, and follow us on our social media channels.
