Skip to main content

One post tagged with "python"

View All Tags

· 6 min read
Krishika Singh

In this blog ,we are going to talk about how easily you can set up your pipeline using YAML.

Harness includes visual and YAML editors for creating and editing Pipelines, Triggers, Connectors, and other entities. Everything you can do in the visual editor you can also do in YAML.

For detailed information about using Harness YAML visit Harness YAML Reference and Harness YAML Quickstart.

Before we begin

Make sure you have the following set up before you begin this tutorial:

  • GitHub Account: This tutorial clones a codebase from a Github repo. You will need a GitHub account so Harness can connect to GitHub.
  • Docker Hub account and repo: You will need to push and pull the image you build to Docker Hub. You can use any repo you want, or create a new one for this tutorial.

Getting Started

  • Fork the repository

    For this demo, we are using Python-pipeline-samples.

  • Login into Harness UI

    • Go to Harness.

    • Sign up for the Harness platform.

    • Once you signup you will enter the Harness UI as shown below.

    • Go to Builds and select Create a Project.

      • Give the name of the Project -> `Save and Continue
      • You can also invite collaborators, it's optional.
    • After Save and Continue select the module as `Continous Integration.

      After selecting the module as Continous Integration you will see the screen as shown in the below screenshot.

    • Select Create a Pipeline.

      • Name your Pipeline.
      • Choose the setup as Inline.
      • Select Start. Refer to the below screenshot:

Getting Started

  • After the Creation of the Pipeline, you will enter the pipeline studio as shown below
  • As you can see in the pipeline studio we have two options, one is VISUAL and the other is YAML. Navigate to the YAML editor, as shown below.
  • Copy and Paste the below YAML file into the editor.

Note:- Paste the below YAML file just below the tags{}.

  properties:
ci:
codebase:
connectorRef: <+input>
build: <+input>
depth: <+input>
prCloneStrategy: <+input>
stages:
- stage:
name: build test and run
identifier: build_test_and_run
type: CI
spec:
cloneCodebase: true
infrastructure:
type: KubernetesHosted
spec:
identifier: k8s-hosted-infra
execution:
steps:
- step:
type: Run
name: Code compile
identifier: Code_compile
spec:
connectorRef: <+input>
image: python:3.10.6-alpine
shell: Sh
command: python -m compileall ./
- step:
type: Run
name: Create dockerfile
identifier: Create_dockerfile
spec:
connectorRef: <+input>
image: alpine
shell: Sh
command: |-
touch pythondockerfile
cat > pythondockerfile <<- EOM
FROM python:3.10.6-alpine
WORKDIR /python-pipeline-sample
ADD . /python-pipeline-sample
RUN pip install -r requirements.txt
CMD ["python3" , "./app.py"]
EOM
cat python-docker file
- step:
type: BuildAndPushDockerRegistry
name: Build and Push an image to the docker registry
identifier: Build_and_Push_an_image_to_docker_registry
spec:
connectorRef: <+input>
repo: <+input>
tags:
- latest
dockerfile: pythondockerfile
optimize: true
variables:
- name: container
type: String
description: ""
value: docker
- stage:
name: Integration test
identifier: Integration_test
type: CI
spec:
cloneCodebase: true
infrastructure:
useFromStage: build_test_and_run
execution:
steps:
- step:
type: Background
name: "python server "
identifier: python_server
spec:
connectorRef: <+input>
image: <+input>
shell: Sh
command: python3 ./app.py
- step:
type: Run
name: "test connection to server "
identifier: test_connection_to_server
spec:
connectorRef: <+input>
image: curlimages/curl:7.73.0
shell: Sh
command: |-
sleep 10
curl localhost:5000

  • Click on Save.

    Navigate to VISUAL and now you can see your two-stage pipeline ready as shown below in the screenshot. That's the beauty of YAML in Harness.

    You can navigate through all the steps in the pipeline and explore the pipeline.

Inputs

Before running the pipeline, let's create a GitHub and Docker connector.

  • GitHub Connector

    Under Project setup select Connectors.

    Click on + New Connector

    Select Code Repositories and Choose Github.

    You can refer to the below screenshot.

    Change the Connector settings as follows:

    1. Overview

      Name: python-sample-connector

      Select Continue.

    2. Details

      URL Type: Repository

      Connection Type: HTTP

      GitHub Repository URL: Paste the link of your forked repository

      Select Continue.

    3. Credentials

      Username: (Your Github Username)

      Personal Access Token: Check out how to create personal access token

      Secret Name: Git-Token

      Secret Value: PAT value generated in Github

      Select Enable API access (recommended)

      Under API Authentication-> Personal Acess Token select the name of the secret created in the previous step.

      Select Continue.

    4. Select Connectivity Mode

      Under Connect to the provider-> Select `Connect through Harness Platform.

      Select `Save and Continue.

    5. Connection Test

      You will see Verification Successful which means your connector is connected successfully to your codebase.

      For reference, you can also check out this video on our Harness Community youtube channel

      To develop more understanding of Connectors [check out the docs here](https://docs.harness.io/category/o1zhrfo8n5-connectors)
  • Create a Docker Connector

    Under Project setup select Connectors.

    Click on + New Connector

    Select Artifacts Repositories and Choose `Docker Registry.

    You can refer to the screenshot below

    Change the settings as follows

    1. Overview Name- docker quickstart

    2. Details

      • Docker registry URL - https://index.docker.io/v1/
      • Provider type - Docker Hub
      • Authentication - `Username and Password
      • Username - Docker hub username
      • Secret Token - Check out how to create docker PAT
    3. Select Connectivity Mode

        Under `Connect to the provider`-> Select `Connect through Harness Platform.

      Select `Save and Continue.

      For your reference you can also check out this video on our Harness Community YouTube channel:

  • Create a Docker Repository

    1. Log in to Docker Hub
    2. Go to Repositories -> Select Create Repositories.
    3. Give a name to your repository and you can choose whether you want you repo to be public or repo.

Run the Pipeline

Navigate back to the Pipeline studio and click on Run.

On Clicking, you will see a page asking for inputs so as to run the pipeline, you can refer to the below screenshot

  1. CI Codebase

    • Connector- Select the Github Connector you created in the previous step.
  2. Stage: build test and run

    Step: Code compile

    • Container Registry- Select the Docker Connector you created in the previous step.

    Step: Create dockerfile

    • Container Registry- Select the Docker Connector.

    Step: Build and Push an image to Docker Registry

    • Docker Connector- Select the Docker Connector.
    • Docker Repository- docker-hub-username/repository-name
  3. Stage: Integration Test

    Execution

    Step: python server

    • Container registry- Select the Docker Connector.
    • Image- docker-hub-username/repository-name

    Step: test connection to the server

    • Container registry- Select the Docker Connector.

Click on Run Pipeline.

It will take around less than 3 mins to execute your Pipeline.

After successful completion and execution of all the steps you will see something similar to this:

This article explained YAML based onboarding process, if you want to try out Harness UI based onboarding do check out this tutorial:-