Firebase: Accessing Firestore and Firebase through a proxy server
Enterprise developers often have to deploy their applications behind a corporate HTTP proxy in order to meet various security and IT compliance requirements. This means applications that depend on Google Cloud Platform (GCP) libraries and Firebase Admin SDKs should be configured to make all outgoing service requests through a proxy. In this post we explore how to achieve this in several programming languages. We will focus on Firebase services that are typically accessed through a REST interface, as well as Cloud Firestore, which is accessed over gRPC. We leave Firebase Realtime Database out of the discussion for the most part, but if that’s what you care about, jump ahead towards the end of the post where I have remarked on that topic.
Trying out the samples
If you wish to try out the examples in this post, you will need an HTTP proxy server to test against. Execute the following command in a shell to get a Docker container with a Squid proxy up and running in your local machine.
$ docker run -p 3128:3128 -ti docker.io/salrashid123/squidproxy /bin/bashThis command starts the container, and provides you a with bash shell with root access. It also maps the port 3128 of the host to the port 3128 of the container. Now you can run the following command in the container’s shell to fire off the Squid proxy.
$ root@2e57029388c0:/apps# /apps/squid/sbin/squid -NsY -f /apps/squid.conf.forward &When started, the proxy server can be accessed via http://localhost:3128 anywhere in your host. Once you are finished, you can simply exit the container’s shell to terminate the container.
Huge thanks to salmaan rashid for his excellent article on this subject. The above instructions and the Docker image are courtesy of him.
The test application
Our test application depends on two cloud services. It…
- queries Cloud Firestore to find the name of a topic, and
- sends a notification to the topic via Firebase Cloud Messaging (FCM).
The application uses Google application default credentials (ADC) to authorize remote service calls. Credentials also call out to a remote service to obtain OAuth2 access tokens. Therefore technically our application depends on three remote services. We will need to forward these token fetch requests via the HTTP proxy as well.
If you’re following along, set up your environment for ADC by downloading a service account JSON file from your Firebase project, and setting the following environment variable to point to it.
$ export GOOGLE_APPLICATION_CREDENTIALS=path/to/serviceAccount.jsonThe rest of this article assumes you are familiar with at least one of the Firebase Admin SDK implementations, and you have some experience writing code with Firebase APIs. If you’re new to all that, go through the Admin SDK getting started guide first.
You will also need some sample data in your Firestore database to run these examples. Go ahead and create a collection named users in Firestore using the Firebase console. Add a document named alice to it. This document should have a topicName field with the string value test-topic.
Go
The Go Admin SDK is one of the easiest to deploy behind a proxy. Simply set the proxy server URL as an environment variable, and you are done.
$ export HTTPS_PROXY=http://localhost:3128You can try this out by executing listing 1, which invokes both Firestore and FCM APIs.
If you inspect the access log of the Squid proxy while executing listing 1, you will see something like the following. The Squid access log can be found in the /apps/squid/var/log directory of the Docker container.
1544900567.546 391 172.17.0.1 TCP_TUNNEL/200 3606 CONNECT accounts.google.com:443 - HIER_DIRECT/64.233.177.84 -
1544900568.448 437 172.17.0.1 TCP_TUNNEL/200 4476 CONNECT fcm.googleapis.com:443 - HIER_DIRECT/172.217.0.42 -
1544900568.768 1826 172.17.0.1 TCP_TUNNEL/200 4160 CONNECT firestore.googleapis.com:443 - HIER_DIRECT/172.217.6.42 -These are the three remote service calls made by our test application. The call to accounts.google.com is for fetching an access token. The subsequent calls are for Firestore and FCM. It appears that the SDK holds the gRPC connection to Firestore open until the program exists, and therefore it gets logged last.
Java
There are couple of different ways to deploy the Java Admin SDK behind a proxy server. The easiest way is to configure proxy settings via system properties. This is illustrated in listing 2.
We first specify the proxy settings for the JVM by setting the https.proxyHost and https.proxyPort system properties. These two properties are sufficient to get the Firestore calls routing through our proxy server. The third system property instructs the default HTTP transport used by the Admin SDK to refer to the same proxy settings configured at the JVM-level. This transport (NetHttpTransport) is what handles the REST calls made by GoogleCredentials and FirebaseMessaging. The system properties in listing 2 are hard-coded into the program. You may instead take them out of the code, and pass them as JVM flags (-Dhttp.proxyHost and -Dhttp.proxyPort).
If you need a bit more control over how proxy settings are configured, the Firebase Admin SDK allows you to specify a custom HTTP transport with proxy settings as shown in listing 3.
In lines 18–20 we initialize a custom NetHttpTransport instance with a proxy configuration. This gets passed into both GoogleCredentials and FirebaseOptions where it is used by the SDK when making REST calls. However, Firestore Java client only supports configuring proxy settings via system properties at the moment. Therefore the configuration in listing 3 does not affect Firestore API calls.
Node.js
To use the Admin Node.js SDK behind a proxy, you must use the latest versions of certain dependencies. Specifically, you need:
firebase-adminv6.4.0 or higher, andgtokenv2.3.1 or higher.
Typically, you get both if you just install the latest version of firebase-admin. You can run the following NPM command as a sanity check to ensure that the required package versions have been installed:
$ npm ls gtoken
/home/hiranya/Projects/firebase/proxy-config/node-example
└─┬ [email protected]
└─┬ @google-cloud/[email protected]
└─┬ [email protected]
└─┬ [email protected]
└── [email protected]Older versions of gtoken do not support SSL tunneling via proxy servers due to a bug (see GitHub issue). If NPM has installed an old version into your project by any chance, you can upgrade it as follows:
$ npm install [email protected]Having verified you have the right dependencies, set the following environment variable to enable proxy support for Firestore (Notice the lower-cased variable name):
$ export https_proxy=http://localhost:3128Finally, to enable proxy support for other APIs like FCM, you can use the newly introduced httpAgent option in the Node.js Admin SDK. Listing 4 shows what the resulting code would look like.
In lines 6 and 7 we set an HTTP Agent to be used by the credentials, and the other modules in the SDK. This ensures that both OAuth2 token retrieval and the subsequent calls to FCM get routed through the proxy server.
When you inspect the Squid access logs produced by running the above example, you will see four entries instead of three. This is because the Node.js Firestore client uses a different library to fetch OAuth2 access tokens — namely gtoken. As a result the application makes an additional token request to https://www.googleapis.com/oauth2/v4/token.
1544935728.247 353 172.17.0.1 TCP_TUNNEL/200 4659 CONNECT www.googleapis.com:443 - HIER_DIRECT/172.217.12.106 -
1544935728.835 175 172.17.0.1 TCP_TUNNEL/200 3710 CONNECT accounts.google.com:443 - HIER_DIRECT/216.58.195.77 -
1544935729.356 519 172.17.0.1 TCP_TUNNEL/200 4443 CONNECT fcm.googleapis.com:443 - HIER_DIRECT/172.217.164.106 -
1544935729.360 1584 172.17.0.1 TCP_TUNNEL/200 4501 CONNECT firestore.googleapis.com:443 - HIER_DIRECT/172.217.6.42 -Listing 4 uses a third-party package named https-proxy-agent to initialize an HTTP Agent suitable for our use case. You are free to use any package you wish for this purpose. I have also experimented with the tunnel2 package with positive results.
Python
Python Admin SDK is pretty straightforward to deploy behind a proxy. Simply set the https_proxy environment variable as shown below.
$ export https_proxy=http://localhost:3128Notice the lower-cased variable name. This instructs both the Firestore Python client, and the requests library (which is what the Admin SDK uses to make REST calls) to route outgoing calls via the specified proxy server. Listing 5 shows the application code.
You should see the familiar three log entries in the Squid access log when the above example is executed.
What about the Realtime Database (RTDB)?
The answer depends on the language.
- Python and Go: The RTDB APIs in these SDKs are based on REST. Therefore they accept the same environment variables outlined above, and route the requests through the specified proxy. It’s no different from calling FCM.
- Node.js and Java: These SDKs use WebSockets to communicate with RTDB. Currently there’s no way to configure the SDKs to send this traffic through an HTTP proxy.
Conclusion
A lot of what we have discussed here applies to other Firebase and GCP APIs as well — with RTDB being the primary exception. Firebase Auth, Project Management, and Cloud Storage APIs all follow the same principles, and therefore can be set up in a similar manner to be used behind a proxy. There’s also ongoing work to make this experience better for the developers, and consistent across different libraries. So do keep an eye out for upcoming Firebase releases. You can also help improve the SDKs by providing feedback and code contributions to the open source Firebase SDKs.
Join our community Slack and read our weekly Faun topics ⬇
