Would you like to TALK about it?

Introducing @this-dot/rxidb

October 13, 2022 | Originally published on the ThisDot blog

cover

When we are working on PWAs (Progressive Web Applications), sometimes we need to implement features that require us to store data on our user’s machine. One way to do that is to use IndexedDb. Using the IndexedDb browser API has its challenges, so our team at This Dot have developed an RxJS wrapper library around it. With @this-dot/rxidb, one can set up reactive database connections and manipulate its contents the RxJS way. It also provides the ability to subscribe to changes in the database and update our UIs accordingly.

In this blog post, I’d like to show you some small examples of the library in action. If you’d like to see the finished examples please follow the following links:

You can also check our OSS repository for more examples.

Storing data in chronological order

Imagine that you are working on a special text editor app, or something that needs to keep data between page reloads. These kinds of apps usually need to track larger amounts of data, it would be a bad practice to use localStorage for that. In the following example, we will focus on how to store and delete rows in an Object Store that has autoIncrement enabled. For the sake of simplicity, every time the user presses the Add Item button, a timestamp of the event will be stored in the database.

We would also like to be able to remove items from the beginning and the end of this store. We will add two buttons to our UI to deal with that, and we would like them to be disabled if there are no entries in the store. We have a starter HTML that looks like the following:

<h1>@this-dot/rxidb autoincrement example</h1>
<br>
<button id="add-item-btn"> Add item </button>
<button id="remove-first-item-btn"> Remove first item </button>
<button id="remove-last-item-btn"> Remove last item </button>

<hr>
<div id="container"></div>

Initiating the database

For us to be able to store data in IndexedDb, we need a database connection and an Object Store set up. We want this Object Store to automatically increment, so we don’t need to manually keep track of the last key. We do want to listen to every update that happens in the database, so let’s set up our listeners for the keys and the key-value pairs using the entries() and the keys() operators.

import {
  addItem,
  connectIndexedDb,
  deleteItem,
  entries,
  getObjectStore,
  keys,
} from '@this-dot/rxidb';

// ...

const DATABASE_NAME = 'AUTO_INCREMENT';

const store$ = connectIndexedDb(DATABASE_NAME).pipe(
  getObjectStore('store', { autoIncrement: true })
);

const keyValues$: Observable<{ key: IDBValidKey; value: unknown }[]> =
  store$.pipe(entries());
const keys$: Observable<IDBValidKey[]> = store$.pipe(keys());

We want to display the contents of the database when they get updated in the #container div. For that, we need to subscribe to our keyValues$ observable. Whenever it emits, we want to update our div.

const containerDiv: HTMLElement = document.getElementById(`container`);

// ...

keyValues$.subscribe((entries) => {
  const content = entries
    .map(({ key, value }) => `<div>${key} | ${value} </div>`)
    .join('\n<br>\n');
  containerDiv.innerHTML = content;
});

Manipulating the data

We have three buttons on our UI. One for adding data to the Object Store, and two for removing data from it. Let’s set up our event emitters, using the fromEvent observable creator function from RxJS.

const removeFirstBtn: HTMLElement = document.getElementById(
  'remove-first-item-btn'
);
const removeLastBtn: HTMLElement = document.getElementById(
  'remove-last-item-btn'
);
const addItemBtn: HTMLElement = document.getElementById('add-item-btn');

const addItemBtnClick$ = fromEvent(addItemBtn, 'click');
const removeFirstItemBtnClick$ = fromEvent(removeFirstBtn, 'click');
const removeLastItemBtnClick$ = fromEvent(removeLastBtn, 'click');

We can use the addItem operator to add rows to an automatically incrementing Ojbect Store. When the Add Item button gets clicked, we want to save a timestamp into our database.

addItemBtnClick$
  .pipe(
    map(() => new Date().getTime()),
    switchMap((timestamp) => store$.pipe(addItem(timestamp)))
  )
  .subscribe();

Removing elements from the database will happen on the other two button clicks. We need the keys$ observable, so we can delete the first or the last items in the store.

removeFirstItemBtnClick$
  .pipe(
    withLatestFrom(keys$),
    switchMap(([, keys]) =>
      store$.pipe(
        filter(() => !!keys.length),
        deleteItem(keys[0])
      )
    )
  )
  .subscribe();

removeLastItemBtnClick$
  .pipe(
    withLatestFrom(keys$),
    switchMap(([, keys]) =>
      store$.pipe(
        filter(() => !!keys.length),
        deleteItem(keys[keys.length - 1])
      )
    )
  )
  .subscribe();

Toggling button states

The last feature we want to implement is toggling the remove buttons’ disabled state. If there are no entries in the database we disable the buttons, if there are entries we enable them. We can easily listen to the keyValues$ with the tap operator

const keyValues$: Observable<{ key: IDBValidKey; value: unknown }[]> =
  store$.pipe(
    entries(),
    tap(toggleRemoveButtons)
  );

// ...

function toggleRemoveButtons(
  keys: { key: IDBValidKey; value: unknown }[]
): void {
  if (keys.length) {
    removeFirstBtn.removeAttribute('disabled');
    removeLastBtn.removeAttribute('disabled');
  } else {
    removeFirstBtn.setAttribute('disabled', 'true');
    removeLastBtn.setAttribute('disabled', 'true');
  }
}

Real-world use cases for autoIncrement Object Stores

An automatically incrementing object store could be useful when your app needs to support offline mode, but you also need to log certain events happening on the UI to an API endpoint. Such audit logs must be stored locally and sent when the device comes online the next time. When the device goes offline, every outgoing request to our logging endpoint can instead put the data into this Object Store, and when the device comes online, we just read the events and send them with their timestamps.

Storing objects

Have you ever needed to fill out an extremely long form online? Maybe even that form was part of a wizard? It is a very bad UX when you accidentally refresh or close the tab, and you need to start over. Of course, it could be implemented to store the unfinished form in a database somehow, but that would mean storing people’s sensitive PII (Personal Identifiable Information) data. IndexedDb can help here as well because it stores that data on the user’s machine.

In the following example, we are going to focus on how to store data in specific keys. For the sake of simplicity, we set up some listeners and automatically save the information entered into the form. We will also have two buttons, one for clearing the form and the other for submitting. Our HTML template looks like the following:

<div id="app">
  <h1>@this-dot/rxidb key-value pair store example</h1>

  <hr />

  <h2>Address form</h2>
  <form id="example-form" method="POST">
    <label for="first-name">First Name:</label>
    <br />
    <input id="first-name" required placeholder="John" />
    <br />
    <br />
    <label for="last-name">Last Name:</label>
    <br />
    <input id="last-name" required placeholder="Doe" />
    <br />
    <br />
    <label for="city">City:</label>
    <br />
    <input id="city" required placeholder="Metropolis" />
    <br />
    <br />
    <label for="address-first">Address line 1:</label>
    <br />
    <input id="address-first" required placeholder="Example street 1" />
    <br />
    <br />
    <label for="address-second">Address line 2 (optional):</label>
    <br />
    <input id="address-second" placeholder="4th floor; 13th door" />
    <br />
    <br />
    <div style="display: flex; justify-content: space-between; width: 153px">
      <button id="clear-button" type="button">Clear form</button>
      <button id="submit-button" type="submit" disabled>Submit</button>
    </div>
  </form>
</div>

Based on the above template we do have a shape of how the object that we would like to store would look like. Let’s set up a type for that, and some default constants.

type UserFormValue = {
  firstName: string;
  lastName: string;
  city: string;
  addressFirst: string;
  addressSecond: string;
};

const EMPTY_FORM_VALUE: UserFormValue = {
  firstName: '',
  lastName: '',
  city: '',
  addressFirst: '',
  addressSecond: '',
};

Set up the Object Store and the event listeners

Setting up the database is done similarly as in the previous example. We open a connection to the IndexedDb and then create a store, but this time we just create a default store. This way we have full control over the keys. With this form, we want to write the value of the USER_INFO key in this Object Store. We also want to get notified when this value changes, so we set up the suerInfo$ observable using the read() operator.

import {
  connectIndexedDb,
  deleteItem,
  setItem,
  read,
  getObjectStore,
} from '@this-dot/rxidb';
// ...

const DATABASE_NAME = 'KEY_VALUE_PAIRS';
const FORM_DATA_KEY = 'USER_INFO';

const store$ = connectIndexedDb(DATABASE_NAME).pipe(getObjectStore('store'));
const userInfo$: Observable<UserFormValue | null> = store$.pipe(
  read(FORM_DATA_KEY)
);

To be able to write values into our Object Store and update the data on our UI, we need some HTML elements. We set up constants that point towards our form, the two buttons and all of the inputs inside the form.

const exampleForm = document.getElementById('example-form') as HTMLFormElement;
const submitButton = document.getElementById(
  'submit-button'
) as HTMLButtonElement;
const clearButton = document.getElementById(
  'clear-button'
) as HTMLButtonElement;

const firstNameInput = document.getElementById(
  'first-name'
) as HTMLInputElement;
const lastNameInput = document.getElementById('last-name') as HTMLInputElement;
const cityInput = document.getElementById('city') as HTMLInputElement;
const addressFirstInput = document.getElementById(
  'address-first'
) as HTMLInputElement;
const addressSecondInput = document.getElementById(
  'address-second'
) as HTMLInputElement;

And finally, we set up some event listener Observables to be able to act when an event occurs. Again, we use the fromEvent creator function from rxjs.

const formInputChange$ = fromEvent(exampleForm, 'input');
const formSubmit$ = fromEvent(exampleForm, 'submit');
const clearForm$ = fromEvent(clearButton, 'click');

Set up some helper methods

Before we set up our subscriptions, let’s think through what behaviour we want with this form and the buttons.

It is certain, that we need a way to get the current value of the form that matches the UserFormValue type. We also want to set the input fields of the form, especially when we reload the page and there is data saved in our Object Store. If there is no value provided to this setter method, it should use our predefined EMPTY_FORM_VALUE constant.

function getUserFormValue(): UserFormValue {
  return {
    firstName: firstNameInput.value,
    lastName: lastNameInput.value,
    city: cityInput.value,
    addressFirst: addressFirstInput.value,
    addressSecond: addressSecondInput.value,
  };
}

function setInputFieldValues(value: UserFormValue = EMPTY_FORM_VALUE): void {
  firstNameInput.value = value.firstName || '';
  lastNameInput.value = value.lastName || '';
  cityInput.value = value.city || '';
  addressFirstInput.value = value.addressFirst || '';
  addressSecondInput.value = value.addressSecond || '';
}

The UI should block the user from certain interactions. The submit button should be disabled while the form is invalid, and while a database write operation is still in progress. For handling the submit button state we need two helper methods.

function disableSubmitButton(): void {
  submitButton.setAttribute('disabled', 'true');
}

function removeSubmitButtonDisabledIfFormIsValid(): void {
  const isFormValid = exampleForm.checkValidity();
  if (isFormValid) {
    submitButton.removeAttribute('disabled');
  }
}

Now we have every tool that we need to implement the logic.

Setting up our subscriptions

We would like to write the form data into the Object Store as soon as the form changes, but we don’t want to start such operations on every change. To mitigate this, we are going to use the debounceTime(1000) operator, so it waits for 1 second before starting the write operation. We use our getUSerFormValue() helper method to get the actual data from the input fields and we use the setItem() method on the store$ observable, inside a switchMap operator to write the values. We also want to disable the Submit button when the form changes and re-enable it if the form is valid and the write operation is finished.

formInputChange$
  .pipe(
    tap(() => disableSubmitButton()),
    debounceTime(1000),
    map<unknown, UserFormValue>(getUserFormValue),
    switchMap((userFormValue) =>
      store$.pipe(setItem(FORM_DATA_KEY, userFormValue))
    ),
    tap(() => removeSubmitButtonDisabledIfFormIsValid())
  )
  .subscribe();

We also want to set the values of the input fields, for example, when we refresh the page. We also handle the submit button state and we set the values only if there are values to set. We use our setInputFieldValues() method to update the UI.

userInfo$
  .pipe(
    tap(() => disableSubmitButton()),
    filter((v: UserFormValue | null): v is UserFormValue => !!v),
    tap((storedValue: UserFormValue) => {
      setInputFieldValues(storedValue);
      removeSubmitButtonDisabledIfFormIsValid();
    })
  )
  .subscribe();

When we submit the form, we will probably want to do something asynchronously. When that succeeds, we want to clear our Object Store, so we don’t keep the submitted data on our user’s machine. We also want to update the UI and clear the input fields. In this example our form would send a POST request when we press the submit button, therefore, we call event.preventDefault() on the submit event, so we stay on the page.

formSubmit$
  .pipe(
    // We prevent the native HTML submit event to run, so it won't send a post request for the sake of the example.
    tap((event: SubmitEvent) => {
      event.preventDefault();
      disableSubmitButton();
    }),
    map(getUserFormValue),
    // this is the point where we could do anything with the current form values, for example, send them to the server, etc.
    switchMap(() => store$.pipe(deleteItem(FORM_DATA_KEY))),
    tap(() => setInputFieldValues())
  )
  .subscribe();

And when we want to clear the form, we need to do the same with the data stored in our Object Store.

clearForm$
  .pipe(
    switchMap(() => store$.pipe(deleteItem(FORM_DATA_KEY))),
    tap(() => setInputFieldValues())
  )
  .subscribe();

Real-world use cases for Key-Value pair Object Stores

Having persistent forms between page refreshes is just one useful feature you can do with IndexedDb. Our example above is very simple, but you can have a multi-page form. One could store the progress on such forms and allow the user to continue with the forms later. Keeping the constraints of IndexedDb in mind, one very cool feature is storing data for offline use.

This Dot Labs is a development consultancy that is trusted by top industry companies, including Stripe, Xero, Wikimedia, Docusign, and Twilio. This Dot takes a hands-on approach by providing tailored development strategies to help you approach your most pressing challenges with clarity and confidence. Whether it’s bridging the gap between business and technology or modernizing legacy systems, you’ll find a breadth of experience and knowledge you need. Check out how This Dot Labs can empower your tech journey.

Balázs Tápai

Written by Balázs Tápai.
I will make you believe that I'm secretly three senior engineers in a trench-coat. I overcome complex issues with ease and complete tasks at an impressive speed. I consistently guide teams to achieve their milestones and goals. I have a comprehensive skill set spanning from requirements gathering to front-end, back-end, pipelines, and deployments. I do my best to continuously grow and increase my capabilities.

You can follow me on Twitter or Github.