PID tuning using Genetic Algorithm

PID controller is one of the most popular closed-loop controllers which is used in the automation industry. By fine-tuning 3 constants, you are able to achieve a system which is almost free from any errors. 

On the other hand, Genetic Algorithms are being heavily used by various machine learning enthusiasts around the world for building an efficient and robust deep learning network.

1. What’s a control system?

In simple terms, control system takes some sets of inputs, regulates them to derive the desired output and then directs them. Control system is usually of two types: 1) Open-loop and 2) Closed-loop. The only difference being that, in closed-loop, the error is sent into the controller as a feedback signal. In this article, we’ll be focusing on closed-loop control systems.

Fig 1.1 : Skeleton of a closed-loop control system

2. What is a PID controller?

In a PID controller, we calculate an error e(t) as the difference between the desired set-point and the current value(process variable) and pass it as a feedback signal. The error e(t) is then corrected based on the proportional(p),integral(i) and derivative(d) terms.

Fig 1.2 : A PID controller

 Let’s take an example:

Let’s say you want a drone to fly at an altitude of 3.0m(desired setpoint) and you are currently at a height of 1.0m from the ground(process variable i.e the current value). To reach the desired set-point, you’ll have to change the speed of the propellers. This speed will be calculated based upon the output that we get from the PID controller.

Here the error term e(t) = (3.0–1.0) = 2.0 .

Now, the proportional error = e(t),

the integral error = (e(t) + sum of all previous errors)*dt ,

and the derivative error = (e(t)-e(t-1))/dt

and now the propeller speed(prop_speed) will be modified as :

prop_speed=Kp*proportional_error+Ki*integral_error+Kd*derivative_error

This closed control system loop will help us to stabilise the drone at the desired set-point of 3.0m . The error is continuously feed back into the system and prop_speed is set accordingly.

(Note: For simple explanation, I’ve considered only one propeller. It can be replicated for rest 3. Alternately, you can modify the value of the throttle as well.)

But how do these terms help us to reach the desired point? Let’s see.

1. The proportional term (Kp*proportional_error): helps us to reduce the rise time. In simple terms, helps us to reach the desired set point of 3.0m quickly.
2. The integral term(Ki*integral_error): helps us to reduce any steady-state error. In practical life, we can never achieve an error-free system, and this term helps to reduce it as much as possible. For e.g, if our drone has reached 3.3m, then the derivative term will help to reduce the error and bring the drone to an altitude of 3.0 (+/- 0.0002)m.
3. The derivative term(Kd*derivative_error): helps us to prevents any overshoot i.e prevents the drone to fly off to an altitude much away from the desired set point e.g >4.0m

But how do we find the correct constant values(Kp, Ki and Kd)?

There are many ways to find these constants. In this blog, we’ll see how to find the optimal values using GA.

3. What are genetic algorithms?

Genetic algorithm is a random search method that can be used to solve nonlinear system of equations and optimize complex problems. GA uses probabilistic transition rules instead of deterministic rules and handles a population of potential solutions known as individuals or chromosomes that evolve iteratively.

The concept of Genetic algorithms is heavily inspired by Charles Darwin’s theory of natural selection. It states that off-springs will be naturally selected in such a way that only best set off-springs will survive in the consecutive generations. In other words, only the best characteristics of the current generation will be passed onto the next generation. GA’s are used in optimisation problems.

They comprise of the following steps:

1. Create an Initial population
2. Defining a fitness function
3. Selection
4. Crossover
5. Mutation

Creating an Initial Population:

We start by generating a set of random values. Each value in this is known as a gene. A set of genes is referred to as a chromosome, and a bunch of chromosomes create a population.

Fig 1.3 

Fitness Function:

A fitness function is defined to measure the fitness of each chromosome. Speaking in terms of deep learning, the cost function J is the fitness function. Based on the fitness, the probability of the chromosome being selected in the next generation is calculated. The higher the fitness, the higher the probability to be selected in the next generation. Fitness can be defined according to your need. For e.g: In an optimisation problem if you want to minimise the error, you’ll define a function which will be equal to 0.

Selection:

The main idea of selection is to select/generate individuals for the next generation. There a lot of methods for selection, few of them are:

i. Elitism: Selecting the fittest individual directly to the next generation

ii. Replication: Creating a copy of a chromosome and using it in the next generation

iii. Crossover: In crossover, we select a bunch of individuals and select a few genes from each of the chromosomes and exchange them

Mutation:

To ensure diversity and to prevent an early repetition of the chromosomes(pre-mature convergence) we perform mutation for a few genes. In mutation, the bits are flipped.

Fig : 1.4

After performing mutation, we end up generating a new set of population.

All of these 5 steps are continuously looped until we reach our stopping condition. The stopping condition can either be when you reach your desired fitness or the set of chromosomes keep on repeating in successive generations. You can define your own stopping condition.

4. Using GA for tuning a PID controller.

Alright now that we have laid our foundation let’s see how to use GA to find the values Kp, Ki and Kd.

Let’s consider the previous example of the drone. Our goal is to find the values of Kp, Ki and Kd in such a way that the drone reaches the desired set-point of 3.0m in altitude as fast as possible without any overshoot.

Our fitness function will be e(t) = 0 where the function e at any point t is the difference between the present altitude and 3.0m i.e (3-current_altitude).

We’ll develop 3 generations and have a population of 5 in each generation. Our stopping condition will be when we reach the last generation.

1st Generation:

As you can see from the image, we start with a set of random values for our three constants. We set these values in our fitness function and calculate the fitness(error). After calculating the fitness we start our process of selection.

Since the 1st row(c1,c2,c3) had the lowest error, we directly select it to our 2nd generation(elitism). After that, we applied a crossover between the 2nd row(d1,d2,d3) and 3rd row(a1,a2,a3) with a cross-over point of 1 . We applied the similar crossover between the 2nd row(d1,d2,d3) and 4th row(b1,b2,b3). Both of these values gave us 4 more values. With the help of these algorithms, we generated our 2nd generation.

2nd Generation:

We apply similar operations to generate the 3rd generation from the 2nd generation.

Note: You can develop your own algorithm to generate new generations. You can set your own threshold value through which you want to apply elitism, replication, crossover etc. A GA usually has a lot of generations with a greater population.

Now that we’ve reached our 3rd generation, our final generation, we stop developing new generations. We’ll calculate the fitness of each chromosome and select the chromosome having the best fitness. In our case, the PID values having the least error.

Conclusion :

To summarise, by iterating through a bunch of generations containing a different set of values and by propagating only the best values to the next generation we end up having a near to perfect PID controller.