Prerequisites and step-by-step guide

Prerequisites

• Red Hat OpenShift cluster.
• Fulfilled the learning exercise Deploying and Troubleshooting Red Hat Developer Hub on OpenShift: A Practical Guide.

Step-by-step guide

1. Configure GitHub

In order to get started with the GitHub integration, we will need some data/credentials from GitHub: a personal access token and a client ID and client secret from a GitHub application. You can work with a personal access token, or you can generate one on a GitHub organization, which is better within a professional environment.

• 1.1 Get the credentials for a GitHub application within a GitHub organization

  There are advantages of working with a GitHub organization: you can better isolate repositories, you can work together with multiple people without having to share credentials, ….

  1. Create a GitHub organization. (I opt for the "Free" one.)

     

     

  2. Fill the form and hit "next"

     

  3. Click "Complete setup"

  4. Go to settings of the organization

     

  5. Scroll down to "Developer Settings" > "GitHub Apps", not to "GitHub Apps" under Third-party access 

     

  6. Create GitHub App:
     1. Name: for us "developer-hub-training-app"
     2. Homepage URL, i have used https://developers.redhat.com/ (you can fill what you want):

     3. Uncheck the "active" box for webhook

        

     4. When creating a GitHub App, you must select permissions (i.e., repository and group permissions, but mainly repository permissions) to define the level of access for the app. The permissions required vary depending on your use of the integration:
        • Use case: Reading software components. Required permissions:
          • Contents: Read-only
          • Commit statuses: Read-only
        • Use case: Reading organization data. Required permissions:
          • Members: Read-only

        • Use case: Publishing software templates. Required permissions:

          Take this one for the rest of this exercise.

          • Administration: Read & write (for creating repositories)
          • Contents: Read & write
          • Metadata: Read-only
          • Pull requests: Read & write
          • Issues: Read & write
          • Workflows: Read & write (if templates include GitHub workflows)
          • Variables: Read & write (if templates include GitHub Action Repository Variables)
          • Secrets: Read & write (if templates include GitHub Action Repository Secrets)
          • Environments: Read & write (if templates include GitHub Environments)

     5. Select "Only this account" for "Where can this GitHub App be installed"

        

     6. Click "Create GitHub App".

     7. Get your client and app ID and store it somewhere where you can access it later on.

        

     8. Generate a new client secret.

        

     9. Store the client secret somewhere where you can access it later on.

        

     10. Click "save changes"

     11. Generate a private key. It will trigger the download of .pem file. Store it somewhere where you can access it later on.

         

     12. Install the application

         

     13. Click "install"

         

2. Create a basic GitHub integration within Developer Hub (i.e., repository creation and scanning)

In order to create the GitHub integrations, we will start from the configuration that we ended up with in the previous learning exercise.

In case you don’t want to run them manually, you can execute the following commands to get in the right state (Please ensure you modify secrets_rdhd-secret.yaml file with your cluster config as mentioned in the previous exercise before running following command):

 cd setting-up-developer-hub-through-the-operator/
 sh cheat/run_all.sh

The required manifests are available in the Developer Hub training exercises repository on GitHub. For this exercise, all manifests are in the folder ‘github-integrations’, so check it out before going further.

• 2.1 Log in with OpenShift

• As described in the last training exercise, you have the option to work with the UI (i.e., the OpenShift console) or with the command line interface (i.e., oc). Within this workshop I will opt for the command line interface, but if you’d like to work with the UI, feel free to check out the previous exercise. (In case you want to work with the UI, because you can’t install the oc command line tool, then you can check out OpenShift dev spaces as well. It will give you an IDE, accessible from the browser, running on OpenShift and already connected to the cluster).

• 2.2 Add secrets and GitHub info into a secret

• Within GitHub, we already created an organization and a GitHub app within that organization. For that GitHub application, we got an application ID, a client ID, a client secret and a private key. This is all information that we will need to pass to our Developer Hub deployment. As it is not safe to put this directly in the application configuration, we will opt for storing it in a secret and linking the secret to the application configuration afterward. 

  In order to get the private key, open the .pem file that was downloaded when you generated a private key:

  

  You can paste the entire block, with the start and the end line, in the secret file secrets_github-credentials.yaml.

  You can create the secret by applying the following YAML definition.

  !!! Important to note is that you will have to add your credentials to this YAML before applying it. Next to that, make sure that the private key value is aligned properly.

  

  oc apply -f manifests/secrets_github-credentials.yaml

  Expected output:

  

• 2.3 Create a dynamic plugins configuration

• Dynamic plugins in Red Hat Developer Hub enable developers to extend the platform's functionality without modifying the core application. They allow users to create and integrate custom features, tools, or workflows that dynamically load into the platform at runtime. This approach ensures flexibility, as plugins can be updated, added, or removed independently of the core Developer Hub, enabling organizations to tailor the developer experience to their specific needs while maintaining seamless scalability and modularity.
• In order to enable GitHub integration, we will need to add a dynamic plugin for these integration(s) too. This can be done by applying the following YAML definition.
• You can create the secret by applying the following YAML definition.

• oc apply -f manifests/dynamic-plugins-v1.yaml

• Expected output:

  

• 2.4 Create an application config configuration

  In Red Hat Developer Hub, the app-config.yaml file is the central configuration file that defines the behavior, integrations, and settings of the Developer Hub instance. It manages the platform’s core functionality and plugins by specifying:

  1. Backend Services and Integrations: Configures APIs, authentication mechanisms, and connections to external systems (e.g., GitHub, Kubernetes, CI/CD tools).
  2. Plugins and Features: Enables or disables plugins and custom features within the hub.
  3. Entity Providers: Defines sources for cataloging software entities, such as Git repositories or cloud platforms.
  4. UI Customization: Allows basic customization of themes, navigation, and branding.
  5. Runtime Behavior: Sets parameters for caching, logging, and other operational aspects.

  By centralizing configuration, the app-config.yaml file ensures consistency, making it easy to manage, extend, and adapt the Developer Hub to an organization’s evolving requirements.

  In order to enable GitHub integration, we will need to create an app-config file after adding the dynamic plugin for these integration(s). The dynamic plugin will enable the underlying dependency, the config within the app-conf file will configure this dependency. Adding the basic GitHub integration configuration can be done by applying the following YAML definition.

  oc apply -f manifests/app-config-v1.yaml

  Expected output:

  

  2.5 Add the dynamic plugins and the application configuration to the instance configuration

• Now that we have the secrets configured, the dynamic plugins listed, and the application configuration configured, we just need to glue everything together within our Developer Hub instance configuration. This can be done by applying the following YAML definition.
• (we will be extending the instance configuration from previous workshop)

• oc apply -f manifests/instance-config-v1.yaml

• Expected output:

  

• You can check the status of the newly created pod by running the following command (the newly created pod should get the status ‘Running’):

• oc get pods -n demo-project -o json | jq '.items[] | 
  select(.metadata.ownerReferences[0].name | 
  startswith("backstage-developer-hub-")) | {POD: .metadata.name, DEPLOYMENT: 
  .metadata.ownerReferences[0].name, Status: .status.phase, StartTime: 
  .status.startTime }'

• Important note: this configuration should be the last step, or you will have to delete the instance and reinstall it through the YAML definition, as it will crash for secrets or configs not found.

  

  Now we will wait until the pods restart and we will access Developer Hub. If you are not able to access Developer Hub in this stage, then something went wrong configuration wise. You can rely on the "how to debug" section of previous learning exercise to check what went wrong.

  For me, it started without errors, so let’s go to the next session and try the GitHub integrations. You should be able to see

  "class":"GithubEntityProvider$1","level":"info","message":"Read x GitHub repositories (x matching the pattern)"

  

  In case you’re facing "github-provider:providerId refresh failed, HttpError: API rate limit exceeded, you’re required to log in", check if the organization name is correct and that your secret values are mapped correctly. I faced the issue when I made a copy-paste error on the organization name in the app-config file.

3. Try-out the GitHub integration

• 3.1 Create a repository via a software template

• The golden path template (i.e. software template) in Developer Hub is a standardized blueprint that helps teams create new projects or services consistently and efficiently. These templates provide pre-configured best practices, coding standards, and integrations tailored to an organization’s needs. Developers can use them to quickly scaffold projects, such as microservices, libraries, or applications, ensuring compliance with organizational guidelines and streamlining onboarding. By promoting reuse and automation, golden path templates accelerate development and reduce cognitive load for developers. 
• As a first integration, we will initiate a golden path, also known as a software template. In order to do so, go to "create". When you are logged in with admin permissions, you will have the possibility to add templates to Developer Hub from within this screen.

• 

  As for now, we have the opportunity to manually add a software template or to configure it via the app-config file. In order to add it via the app-config file, you can apply the following YAML definition:

  oc apply -f manifests/app-config-v2.yaml

  Expected output:

  

  In the updated YAML, I added ‘added to enable software templates’ section to show which configuration was added. It is mainly a location reference and a rule to enable certain catalog entities in Developer Hub (of which Template is one). More on catalog entities in later training exercises. This configuration will make use of the GitHub integrations configuration to enable repository creation in GitHub. It can take a couple of minutes, depending on the processing interval you configured for the template to be imported. As described in the previous exercise, you can monitor this in the backstage_plugin_catalog database.

  When you execute the debug query, you should be seeing something like this:

  

  Now that we have the template, we can initiate it.

  • Go to the software template catalog, select the template "Default Quarkus Application" and click "choose".

    

  • Fill the section "Repository Information":

    • Repository name: I opt for "first-initiated-software-template".

      (I will reuse it as a component name, which will be the name in Developer Hub).

    • Repository owner: The GitHub owner of the repository. For me, it is "maarten-dev-hub-training-organization". You can find it back by going to your repositories in GitHub: E.g., for 'https://github.com/maarten-dev-hub-training-organization/initial-test-repo', it is 'maarten-dev-hub-training-organization'.

    

  • Fill the section "Component Information":

    • Name: The name of the component in GitHub. I opt for "first-initiated-software-template-component".
    • Owner: The owner within Developer Hub. For now, we didn’t configure any owners, so just fill in a random name. I opt for "unset-owner". When you enter it, it becomes ‘group:default/unset-owner’ automatically.

    

  • Click "review".

    

  • Click "create".

    

    The repository should now be created in GitHub.

    

    In real world use cases, you would add an extra configuration in the template definition that directly adds this component to the component catalog, but more on that in the training exercise about software templates (we have that much to say about software templates, that it will get its own exercise). For now, when we want to see it in Developer Hub, we will have to wait for the GitHub provider to import the component in Developer Hub.

    

    As we are a one-person organization at the moment, we will need to restrict the security on the projects a bit, in order to modify them (please, don’t do this in real-world scenarios). In order to achieve this:

  • Go to the repository
  • Go to settings
  • Got to branches

  • Click "edit" on the main branch. 

    

  • Uncheck "Require a pull request before merging". 

    

  • Save changes.

• 3.2 Import repositories from GitHub within Developer Hub

  In Backstage, the GitHub provider facilitates the discovery and importing of repositories from GitHub into the Backstage catalog. It automates the process of identifying repositories, extracting metadata, and registering them as entities in the catalog. 

1. Here's how it works:

   1. Repository Discovery: The GitHub provider scans a GitHub organization or specific repositories based on configuration settings. It identifies repositories that match criteria such as file structures, naming conventions, or the presence of specific files (e.g., catalog-info.yaml).
   2. Metadata Extraction: Once a repository is identified, the provider retrieves the catalog-info.yaml file or other configuration files in the repository. This file contains metadata that defines the entity (e.g., a service, API, or library) to be included in Backstage.
   3. Registration in the Catalog: The provider parses the extracted metadata and registers the entities defined in the repository into Backstage's catalog. This allows teams to view and manage these entities within the platform.
   4. Ongoing Synchronization: The GitHub provider can be configured to periodically synchronize with GitHub, ensuring that any new repositories or updates to existing metadata are reflected in the Backstage catalog.
   5. On-demand import: For manual imports, users can provide a repository URL, and the provider fetches the necessary catalog file to onboard the repository into Backstage.

   This process simplifies the integration of GitHub repositories into Backstage, providing a centralized view of services and infrastructure across teams.

   As we already create a GitHub provider and GitHub integration within the app-config configuration and as we already added the ‘backstage-plugin-catalog-backend-module-github-dynamic’ dynamic plugin, no extra configuration is required to enable the import of repositories.

   In order to test this, you can:

   1. Create a repository in GitHub, let’s name it 'import-demo'. 

      

   2. Add a file catalog-info.yaml and add the content of the file

      setting-up-developer-hub-through-the-operator > files > demo-import-catalog-info.yaml to that file.

      

   3. Commit the changes and wait for the provider to fetch the GitHub data (you can again monitor this on database level or just by refreshing the component catalog view). 

      

• 3.3 Enable GitHub authentication

• Up until now, we worked with guest authentication, which is not something we will do in a production setting. There are quite a few supported user providers like OpenID compliant systems (e.g., Keycloak), GitLab, …. For this exercise, we will stick to GitHub authentication.
• In order to do so, you will need to:
  1. Update your GitHub app configuration

     • Go to your organization’s GitHub Apps and click 'edit'. 

       

     • Fill the callback URL field with:

       https://backstage-developer-hub-demo-project.apps.cluster-8wcdk.8wcdk.sandbox1893.opentlc.com/api/auth/github/handler/frame

       !! Be aware: your base domain will be different from mine. It needs to be the base domain of your Developer Hub instance.

       !! Be aware (II): In case you change clusters, you will need to add an extra callback URL.

       

     • Click 'save changes'

  2. Add a new plugin to the dynamic plugins' configuration. It will be the GitHub org plugin, which is responsible for importing users and groups from GitHub into Developer Hub. You can add it to the dynamic plugins by applying the following YAML manifest: (changes are in between ‘added to enable github authentication’ blocks). (version is v3, to be aligned with the app-config files, v2 doesn’t exist for the dynamic plugins).

     oc apply -f manifests/dynamic-plugins-v3.yaml

     Expected output:

     

  3. Now that we have the new plugin enabled, we will need to add some configuration for it in the app-config file. You can do this by applying the following YAML manifest: (changes are in between ‘added to enable github authentication’ blocks)

     oc apply -f manifests/app-config-v3.yaml

     Expected output:

     

     We didn’t need to add anything regarding the GitHub integration part, as it is already configured for previous sections. What you need to do is:

     • Add GitHubOrg configuration in the providers section.
     • Add GitHub configuration in the auth providers section.
     • Set GitHub as a sign in page.
  4. Wait for the pods to restart.

  5. Now, go to the home page of Developer Hub (or log out as guest) and select GitHub login.

     

  6. Log in with your GitHub account. 

     

  7. And you’ll see that you’re not the guest user anymore. 

     

  8. In case you want to disable the option to log in as a guest user, you can do this by applying the following YAML manifest: (changes are in between ‘added to disable guest authentication’ blocks or highlighted with ‘ #changed to enable github authentication’)

     oc apply -f manifests/app-config-v4.yaml

     Expected output: 

     

     In order to hide the guest login, your environment need to be different from “development”.

     In case you would get an error mentioning "the redirect_uri is not associated with this application", then check the following: 

     

•  

  • Check the “baseUrl” field within your rhdh-secrets configuration. It should be similar to

    "https://backstage-developer-hub-demo-project.apps.cluster-5pq52.5pq52.sandbox502.opentlc.com/".

  • Check if your domain (i.e., maybe you got a new cluster) is still listed in your GitHub app configuration:
    • Go to your organization settings.
    • Go to GitHub apps.

    • Click edit 

      

    • Scroll down in general to the callbacks section and check / edit / add your current domain. 

      

    • Now scroll down to the save button and click "save changes".

• 3.4 Enable GitHub actions

• For now, we will work with public repositories. In case you would have initiated a private repository, you can make it public via repository > settings > general > advanced > change visibility > change to public. In case you are working with private repositories (for now), it can be that the CI tab is showing up, but that it doesn’t contain any workflows. 

  

  

  The default CI infrastructure for GitHub is GitHub Actions. We will embed it in our demo application and make it available within Developer Hub. In order to do so, you will need to:

  1. Extend the demo application (or application on which you want to test it, I will reuse the demo-import application from the previous section) with a github actions manifest. This file will be put in the git repository under .github > workflows > github-actions.yaml (you can choose the name of the file, not the directory structure). You can find an example in my commit. I’ve added a dummy github actions manifest in our repository as well, you can find it under github-integrations > files > .github > workflows > github-actions-demo.yaml. It doesn’t do anything more than metadata logging, but this is an exercise on enabling the GitHub actions view, not on GitHub actions themselves.

  2. Now we need to enable the GitHub actions plugin in Developer Hub. You can do this by applying the following YAML manifest: (changes are in between ‘added to enable github actions’)

     oc apply -f manifests/dynamic-plugins-v5.yaml

     Expected output:

     

  3. If you check the details of the catalog component, you’ll notice that you don’t see any CI-tab at the moment.

     

     

     This is not a bug: In order to enable these tabs, you will need to add annotations to the catalog-info.yaml file of the corresponding project. These annotations can enable plugins (if they are installed and configured properly).

  4. We will now configure our project/repository to enable the CI-tab for GitHub Actions. In order to do so, you will need to add the ‘project-slug’ annotation as shown in github-integrations > files > catalog-info.yaml. The slug exists of the organization name and the project name, separated by a forward slash.

     !!! Important, change the organization name to yours! For me it is 'maarten-dev-hub-training-organization'

     

  5. Whenever you now add this or other changes, GitHub Actions will be run on your project. 

     

  6. In order to make this now available in Developer Hub, we don’t have anything to do besides refreshing the screen: GitHub integrations and provider is required, which we already configured so far, just like the GitHub authentication provider. Important to note is that you will need to have a GitHub authentication provider configured, but it doesn’t need to be the main authentication method. The authentication provider will kick in whenever you want to open the tab: if you’re not logged in yet, it will ask you to log in before you get to see the pipelines.

  7. When you refresh the screen, you should now be able to see the CI-tab with the past GitHub actions and a button to retrigger the pipeline. 

     

Summary

We hope by completing this exercise, you have gained practical experience in enhancing Developer Hub with GitHub capabilities, streamlining your development processes, and fostering a more resilient and efficient DevOps environment. Explore more learning exercises for Red Hat Developer Hub here.