Prerequisites and step-by-step guide

Prerequisites

• Red Hat OpenShift cluster.

Step-by-step guide

1. Install the operator

In OpenShift, an Operator is a method of packaging, deploying, and managing Kubernetes-native applications using automation. Operators extend Kubernetes functionality by encapsulating operational knowledge into custom resources and controllers, enabling automated management tasks like installation, scaling, updates, and self-healing. They act as software "agents" that continuously monitor the state of the cluster and ensure that applications are running as intended, reducing the need for manual intervention and simplifying complex operational workflows.

The Developer Hub operator can be installed through the operator hub or through YAML manifests (i.e., GitOps approach). You can choose your preference as both of them will be described.

1.1. Install the operator manually (i.e., through the UI)

1. Go to the OpenShift console and log in with your kubeadmin credentials (or user credentials if you have sufficient permissions).
2. Create a new namespace (i.e., project) that we will use through the exercises, named demo-project.

   1. Go to projects. 

      

   2. Click create project. 

      

   3. Fill in ‘demo-project’. 

      

   4. Click create.
3. Go to Operator Hub

4. Search for the Red Hat Developer Hub operator

5. Click “install”. For us, we will use the fast channel, and at the time of writing, it results in version 1.3.1.

6. Leave the defaults as is (i.e., Developer Hub needs to be installed over all namespaces and preferably installed in its own namespace, being “rhdh-operator”). You can select a custom namespace, but know that it can start behaving wrongly then.

7. Hit install.
8. Wait for the operator to be installed. You can monitor progress in the installed operators section. This can take a couple of minutes.
9. You now should see that the operator is installed with status “succeeded” in the installed operators section.

1.2. Install through YAML manifests

1. The required manifests are available in the Developer Hub training exercises repository on GitHub. For this exercise, all manifests are in the folder ‘setting-up-developer-hub-through-the-operator’, so check it out before going further.
2. Make sure you have the oc command line tool installed locally and that you are logged in against the cluster or that you can apply YAML manifests through the console. My preference is working with the oc command line tool and executing ‘oc apply -f <filename>’. This can be executed from a folder in a git project, so that you can enable GitOps as well.

3. Install the rhdh-operator and the demo-project namespace by applying the following YAML manifest (the demo-project namespace will be used through the exercise):

   oc apply -f manifests/namespaces.yaml

   Expected output:

   

4. In OpenShift, an Operator Group is a resource that defines the scope of permissions and namespaces where operators can be deployed and managed. Essentially, it determines which namespaces an operator can watch and operate on. Operator groups are useful for configuring operators to run in either a single namespace, multiple namespaces, or cluster-wide, allowing for flexible deployment and operational models. This resource is crucial for managing access and segregation, ensuring that operators have appropriate visibility and control based on the environment’s needs. Install the developer hub operator group and the demo-project operator group, by applying the following YAML manifest:

   oc apply -f manifests/operator-groups.yaml

   Expected output:

   

5. Install the Developer Hub operator, by applying the following YAML manifest:

   oc apply -f manifests/operator.yaml

   Expected output:

   

6. Wait for the operator to be installed. You can monitor progress in the installed operators section.
7. You now should see that the operator is installed with status “succeeded” in the installed operators section.

2. Install Developer Hub instance

Now that we have our operator installed, it is time to install an instance of Developer Hub.

1. Get the base domain of your OpenShift cluster

   • Through the UI:

     If you have the OpenShift console open, you can see the base domain in the URL (i.e., starting from “apps”). In my case, it is “apps.cluster-l2jx7.l2jx7.sandbox714.opentlc.com”.

     

   • Through the oc command line tool:

     You can then swap the value in the secret file via e.g., a shell script. An example of this can be found over here.

     oc get ingresses.config/cluster -o jsonpath='{.spec.domain}';echo

2. We now have our base domain (in my case “apps.cluster-l2jx7.l2jx7.sandbox714.opentlc.com”), so we can create the secret. (As mentioned before, the base domain value can be scripted into the secret definition as well). Next to the base domain, this custom application configuration (i.e., secret) contains a sensitive environment variable, named BACKEND_SECRET. This variable contains a mandatory backend authentication key that Developer Hub uses to reference an environment variable defined in an OpenShift Container Platform secret. To create the secret, we only need to apply the following YAML definition (pay attention to the fact that your base domain should match yours (so you will have to update the file secrets_rdhd-secret.yaml)! & be aware of the namespace in which you apply it!):

   • “basedomain” field: basedomain, suffix of your DNS domain name (e.g., apps.cluster-z54p8.z54p8.sandbox3031.opentlc.com)
   • “baseurl” field: root URL to access the Developer Hub instance (e.g., https://backstage-developer-hub-demo-project.apps.cluster-2pddf.2pddf.sandbox2894.opentlc.com/)

    oc apply -f manifests/secrets_rdhd-secret.yaml

   Expected output:

   

3. In Red Hat Developer Hub on OpenShift, dynamic plugins are modular extensions that enhance the capabilities of the Developer Hub by allowing additional features and tools to be loaded at runtime, without needing to redeploy or restart the main application. These plugins are built as self-contained microservices that connect seamlessly to the Developer Hub via APIs. When the Developer Hub starts, it detects and registers available plugins dynamically, making them immediately accessible to users, allowing for flexible and scalable customization aligned with evolving development needs. You can imagine that when you have a dozen of plugins, that this can slow down the start-up of Developer Hub. In order to speed this up, there is an option to cache these plugins on a persistent volume, so that unchanged plugins do not reinitiate during a restart of Developer Hub. In order to install this plugin, apply the following YAML manifest (be aware of the namespace in which you apply it!):

   oc apply -f manifests/pvc_dynamic-plugin-root.yaml

   Expected output:

   

4. Install Developer Hub instance by applying the following YAML manifest (be aware of the namespace in which you apply it!):

   oc apply -f manifests/developer-hub-instance.yaml

   Expected output:

   The warning can be ignored, as we only configure replicas, which is in the correct section. 

   

• Replicas: 1 ⇒ the amount of instances that will be running
• Route: enabled ⇒ a route (external path) will be generated
• When you are working with self-signed certificates, you can uncomment “name: NODE_TLS_REJECT_UNAUTHORIZED” and “value: '0'”.
• Deployment patch ⇒ this makes sure that Developer Hub uses the persistent volume with the dynamic plugins cache on startup. This will speed up consecutive (re)starts.

5. Wait for the Developer Hub instance to get status “Deployed”. You can find this back in Installed Operators > Developer Hub operator > Red Hat Developer Hub (instances).

6. If something went wrong, you can go into the details of that instance and check the event log or go to workloads > pods and check the log of the developer hub pod (in the demo-project namespace).

7. Now wait for the Developer Hub pod to get started, which can take a couple of minutes in the beginning. You can always check the pod logs to monitor the progress.

8. If you don’t see any logs in the beginning, then a reason can be that you are on the default log (i.e., the one from the instance - backstage-backend), while quite some work is done first in the installation of the dynamic plugins (i.e., install-dynamic-plugins).

9. You can now access Developer Hub now with a route you can find in routes > backstage. 

   

   Copy the value and paste it in your browser. You now should be able to see a screen in which you can enter as a guest user. As we didn’t configure any authentication layers yet, it can be that you will get some authentication error popups when opening Developer Hub. Configuring these (e.g., GitHub, Keycloak, …) will be part of future learning exercises.

   

   

3. Debug Developer Hub

When things go wrong, it is always convenient that you know where to look. Within this exercise, we didn’t cover any integration yet, but I can assure you that when we start building integrations, you will be looking back at this chapter. There are several places where you can find information about Developer Hub instances, being:

1. General OpenShift event list (e.g., when secrets or container images are not found).

2. On the Developer Hub instance details page.

   

   1. In the Events section.

      

   2. In the status field of the YAML description of the instance. 

      

3. You can look into the pods associated with the instance. For us, it was just one instance. Don’t be confused: you’ll see two pods in the screenshot, but one is the Developer Hub instance, the other one is the PostgreSQL database.

   In case you want to enable debug logging, go to app-config file and change the log level to DEBUG instead of INFO.

   

   On pod level, you’ll have the event log and two types of logs:

   1. The dynamic plugin logs (i.e., which run first), which contain the information about the startup of the dynamic plugins.

      

   2. The instance log of Developer Hub during and after startup.

      

4. Last, but not least, you have logging/tracing on database level. For this, you’ll have two options: connect locally to the database or use the pod console in OpenShift.

1. Connect to the database locally:
   1. Make sure you have a PostgreSQL client installed on your machine (i.e., I’m using an IntelliJ plugin)

   2. Make sure you are logged in into the cluster via the ‘oc login’ command.

      Enable port forwarding (i.e. more secure than exposing the database through the internet) by executing the following command:

      oc port-forward $(oc get pod -n demo-project | grep backstage-psql | awk '{print $1}') 5432:5432 -n demo-project

   3. Get the password of the postgres user by executing the following command:

      oc get secret backstage-psql-secret-developer-hub -n demo-project -o jsonpath='{.data.POSTGRES_PASSWORD}' | base64 -d ; echo

   4. You can now connect to the PostgreSQL database. For IntelliJ, it looks like this: 

      

2. If you don’t want to use a local client, you can always use the PostgreSQL pod console.

   1. Go to the OpenShift console and go to the PostgreSQL pod details. 

      

   2. Go to the terminal view. 

      

   3. Connect to the database by running ‘psql -U postgres’. There is no need to pass a password, as that’s already mounted via a secret. With the ‘\l’ command, you can list the databases. 

      

   4.  By using the ‘\c’ command, you can select a database. E.g., ‘\c backstage_plugin_catalog’ command will connect to the database backstage_plugin_catalog.

3. Now that we are connected to the database, we have two interesting queries already:

   Connect to database ‘backstage_plugin_catalog’ (which will hold information about integration statuses). To get the integration statuses (errors can be displayed over here as well), just run the following query (available in Developer Hub training exercises repository on GitHub as well. For this exercise, all scripts are in the folder ‘setting-up-developer-hub-through-the-operator’):

   SELECT entity_id,
         location_key,
         entity_ref,
         errors,
         last_discovery_at AT TIME ZONE 'Europe/Brussels' AS last_discovery_at
   FROM refresh_state
   WHERE 1=1 -- dummy where clause to enable the "AND" clauses later on and play with commenting them out
   -- AND last_discovery_at >= NOW() - INTERVAL '15 minutes'
   AND last_discovery_at AT TIME ZONE 'Europe/Brussels' >= '2024-07-27 12:19:28.943856'
   AND errors <> '[]'
   -- AND entity_ref = 'system:default/maartens-wonderful-system'
   ORDER by last_discovery_at desc
   ;

   When you want to run this query, make sure that the Developer Hub is in a running state (if you start it for the first time, it can take some time to become ready. As long as it didn’t start at least once, it can be that the query is returning zero results):

   

   If you then run the query, you should have this output:

   

Summary

This learning exercise provided a hands-on approach to setting up the Red Hat Developer Hub using the Developer Hub Operator. We covered the installation process for the operator, configuration steps to get the Developer Hub up and running, and essential troubleshooting techniques. Key activities included managing deployment configurations, verifying installation success, and identifying and resolving setup issues. By now, you should feel confident in using the Developer Hub Operator to streamline setup processes and address common challenges effectively. Explore our next learning exercise to learn about GitHub integration with Red Hat Developer Hub.