Load testing your software can be like a public bathroom (though hopefully cleaner). after a plumber is done setting up 5 toilets, they might go up to each one individually and flush it, see that they all work and be happy about a good days work. After some time goes by, eventually you will have two people just happen To flush at the same time, leading to a real crappy situation that you might be stuck cleaning up.


Whether you have a dozen or hundreds of thousands of users hitting your system every minute, it is good to know the load your system can handle. Just doing functional tests might not surface problems that could occur when many users use your product concurrently. Things like memory leaks, cpu usage and response time might be spot on when one request at a time is occurs but with 1000s of users these problems can quickly become apparent. If your product happens to go viral on the internet then you also want to ensure that your environment can scale accordingly. This is the whole point of load testing.


For load testing, this is where [JMeter](https://jmeter.apache.org/) can be handy. JMeter is a free open-source tool that allows you to write scripts that make a specified number of API calls at a particular interval. This is an especially simple way to load test your backend systems and simulate 1000s of users using you system at a time. 


## JMeter Setup and Use
 
 

### JMeter install
1. Download JMeter file from [here](https://jmeter.apache.org/download_jmeter.cgi). 
2. If you have a Mac or Windows get the Zip file and extract it to a directory. 
3. Open your terminal or powershell, go to the directory you downloaded the file to, go into the bin folder of the JMeter directory to downloaded and and run the following command:
```bash
sh jmeter.sh
```
3. The UI for JMeter should now show up.

**Note: The UI is not for the actual load test.**


### Single call JMeter script
1. Create a thread group. Here you can specify the number of thread (users) that you will be running, how long the ramp up will be and how long the entire test will last.

![making thread group in jmeter](/images/jmeter_add_thread_group.png "JMeter Thread Group Creation" )

2. Add an API call to the thread group. 
 
![Adding api call in jmeter](/images/jmeter_add_api_call.png "JMeter API call Creation" )

3. Add ways to display results (called "Listeners") for small tests that are under 1000 users
 * There are quite a few options for listeners but the main ones I use are "View Results Tree" and "Summary Report". 
 * View Results Tree just shows a list of which individual calls failed or passed and if they got redirected or not.
 * Summary Report just shows aggregate results of all the calls made.
 * If you have good telemetry and logging on your systems then this might not be needed.

![Adding api result listener](/images/jmeter_add_listener.png "JMeter adding result listener" )

4. Add headers to request if needed. First add the header configuration element then add the actual keys and values you want as headers

![Adding headers](/images/jmeter_add_header.png "JMeter adding headers" )

![Adding header value](/images/jmeter_add_header_value.png "JMeter adding header value" )


5. Add Assertions for response
 * Using assertions, you can ensure that what is returned from the call is what is expected. For instance, if in the top level of the JSON response you expect a data attribute then you can ensure it will be there with "**`$.data`**" and if the call passes in JMeter then the data attribute is there. 
 * The assertion element should be a child of the request element that it will be used for.

![Adding json assertion](/images/jmeter_add_json_assertion.png "JMeter adding json assertion" )

![Adding json assertion values](/images/jmeter_add_json_assertion_value.png "JMeter adding json assertion values" )

### Double call JMeter script
To add another call to the test, you can either duplicate the one you have before or create a new one from scratch like in the earlier steps.
 * Note that every thread (or "user") will run both of these calls in order and as fast as possible
 * For the example below, this single "user" will make "HTTP Request 1" then immediately make "HTTP Request 2" 

![JMeter running two API calls](/images/jmeter_two_api_calls.png "JMeter running two API calls" )

To simulate a more realistic user you can put a delay in between the two requests. This can be done with different timers, the simplest of which is the "Constant Timer" and is shown below

![JMeter adding constant timer](/images/jmeter_add_constant_timer.png "JMeter adding constant timer" )

With this constant delay in place, a single user will make the first HTTP request, wait 10,000 milliseconds(10 seconds) then make the second request.

![JMeter running two API calls with constant delay](/images/jmeter_constant_delay.png "JMeter running two API calls with constant delay" )


This is just a basic setup to get you started. There are many more settings, elements and variations of all of these and those can be found in the JMeter documentation [here](https://jmeter.apache.org/usermanual/component_reference.html#HTTP_Request)


### Specifying Users, Ramp up and Duration

For load testing, the most important part of the script is the actual number of users and duration of the test. In JMeter these settings can be found in the thread group.

Although the definitions of users, ramp up time adn duration are simple it is very crucial to get these right.
The number of users is how many times your script will be running. If "Infinite" is selected then that will be one call right after the other as fast as possible unless a delay is put in.

![JMeter thread group settings](/images/jmeter_thread_group_numbers.png "JMeter thread group settings" )

The ramp up time is the duration of time it takes or all your users to start.
* If you specify 10 users and a ramp up of 10 seconds then 1 user will start every second. 
* If you specify 1000 users over 5 minutes (300 seconds or 300000 milliseconds) then 1 user will start every 1/3 of a second.
 For the duration, this is how long the entire test will run, including the ramp up time. So if you have a ramp up time of 300 seconds with 1000 users and the duration is 600 seconds (10 minutes), then the script will take the first 5 minutes to ramp up to 1000 simultaneous users then those 1000 simultaneous users will run for another 5 minutes. 

Mort information on this can be found in the JMeter documentation [here](https://jmeter.apache.org/usermanual/build-web-test-plan.html#adding_users)

### Using Variables and Functions
There are ways to use random numbers, variables and functions to simulate more realistic data. Variables can be created with different elements such as the the "User Parameters" element. With this you can define variables and reference them elsewhere in the script by wrapping the variable with **`${some_variable}`**. 

Along with variables, there are also built in functions that can be used with those variables. For example, the __time function can be used to return the current time in milliseconds. It can be referenced like the following:

```bash
${__time(EEE\, d MMM yyyy)}
```

There are many more uses for variables and many more functions so if you would like more info then it can be found [here](https://jmeter.apache.org/usermanual/functions.html).


### JMeter Command-Line Interface (CLI)
Depending on your system, you might be able to use JMeter on a normal computer without the use of a third party tool, such as Flood.io. If you have no need for multiple IP addresses and you want to run 3000 users (threads) then you can probably get away with just using the JMeter CLI. Of course, work your way up to 3000 if you have never load tested your system before. Start at 100, then go to 200, then 400, 800 and so on. Eventually you will want to use Flood.io so as not to overwhelm your computer.

**Before running the script using the CLI, deleting any "Listener" elements will make your script run smoother and more consistently.**

To run JMeter without the use of the UI, which can cause slow downs if running an actual load test, you need to follow these steps:
1. Open a terminal and go to the bin directory of the jmeter tool
2. If you have good telemetry for your systems and do not need the JMeter script to save anything for you then you can simply run the following command

For Windows:
```ps
.\jmeter.bat -n -t .\blog_post.jmx
```

For Mac:
```ps
sh ./jmeter.bat -n -t blog_post.jmx
```
* The **`-t`** flag specifies the filename of the saved test plan, where ever that might be on your computer
* The **`-n`** flag specifies "Non-gui" which means you will be launching the test from the terminal and not using the JMeter UI

3. If you do need to save the results then that can be done using the **`-g`** and/or **`-e -o`** flags.

For Windows:
```ps
.\jmeter.bat -n -t .\blog_post.jmx -e -o .\report\ -g report-file.csv
```

For Mac:
```ps
sh ./jmeter.bat -n -t blog_post.jmx -e -o .\report\ -g report-file.csv
```

* The **`-e -o`** flags specify a directory that an HTML report dashboard will be generated in. It will look nice and have lots of cool information.
* The **`-g`** flag specifies a csv file that all of the individual request info will be saved to in case you need that.

4. After running your test either open the HTML file in the directory or play with the csv file and see your results!

**There are many more CLI flags that can be used so if you want to know more then they can be found [here](https://jmeter.apache.org/usermanual/get-started.html#non_gui)**


## Finding the max load of your production system in a testing environment.

1. If your infrastructure runs on docker, as a lot do, you might have a different number of instances running in your production environment than in the test one.
 
2. Make sure the instances can be scaled and that they are scaled off the right metric. If you have a small internal service that does a small bit of validation then the limiting factor for that instance might be throughput rather than cpu usage.
 
3. You database connection limit might not match your request limit. For instance, if you have a service that makes requests to a db, if it can have 200 network connections open but can only have 10 db connections open, if it gets hit with 20 requests then 10 of those will just wait until the db requests finish. This only gets worse the more requests you get.

4. You might find that instances need to communicate between each other in which case scaling might only cause more complexity and therefore make the system slower. This can especially happen with a CMS.

JMeter can be used to load test your APIs. Here are some things you need to know before starting.

Max out your System With JMeter


LIRC is a way to give a computer running a Linux operating system the capability to read incoming infrared (IR) data. The most useful infrared data that one might want to capture is that of TV remote controls.

## What is LIRC ##

[LIRC](https://lirc.org/) stands for Linux Infrared Remote Control and is a package that allows the decoding of infrared signals of many remote controls. The LIRC package is made to where a remote control can be easily configured then the button pressing data can easily be captured in a serial port that the decoded data is output to.


## LIRC Setup ##

This example is using a [Raspberry Pi 4](https://www.raspberrypi.com/products/raspberry-pi-4-model-b/) and lists out the steps for setting up LIRC hardware and software. 
Assuming you already have a Raspberry Pi 4 you should be able to follow these steps:

* Buy a cheap IR sensor, like [this one](https://www.amazon.com/barterine-KY-022-Infrared-Receiver-Arduino/dp/B01MG1LFA0/ref=sr_1_20?keywords=arduino+ir+receiver+module&qid=1640665624&sprefix=arduino+ir+receiver+%2Caps%2C89&sr=8-20), on Amazon.
* Wire the 5V VCC on the IR sensor to the Raspberry Pi 5V pin, the ground receiver pin to the ground pin on the Raspberry Pi and the receiver's data pin the the Raspberry Pi's GPIO 18 pin as shown in the diagram below.

![wiring diagram for IR sensor to Raspberry Pi](/images/RPI-Infrared-sensor-wiring.jpg)

* Grab your favorite TV remote and fire up your Raspberry Pi.
* The first-time setup for the Pi will not be covered in these steps but if you need help with that then instructions can be found [here](https://www.raspberrypi.com/documentation/computers/getting-started.html).
* With your Raspberry Pi ready to go, open the terminal and run the following command to update and upgrade the OS
```bash 
sudo apt update
sudo apt upgrade
```
* Then install LIRC with the following command
```bash 
sudo apt-get install lirc liblircclient-dev
```
* run the following command to edit the `/etc/modules` file
```bash 
sudo nano /etc/modules
```
* add the following line to the bottom of the  `/etc/modules` file
```bash 
lirc_dev lirc_rpi gpio_in_pin=18
```
* now edit the `/etc/lirc/hardware.conf` file using `sudo nano /etc/lirc/hardware.conf`
* Change the contents of hardware.conf to the following:
```bash 
# /etc/lirc/hardware.conf
#
# Arguments which will be used when launching lircd
LIRCD_ARGS="--uinput"
 
# Don't start lircmd even if there seems to be a good config file
# START_LIRCMD=false
 
# Don't start irexec, even if a good config file seems to exist.
# START_IREXEC=false
 
# Try to load appropriate kernel modules
LOAD_MODULES=true
 
# Run "lircd --driver=help" for a list of supported drivers.
DRIVER="default"
# usually /dev/lirc0 is the correct setting for systems using udev
DEVICE="/dev/lirc0"
MODULES="lirc_rpi"
 
# Default configuration files for your hardware if any
LIRCD_CONF=""
LIRCMD_CONF=""
```
* Now run the following command and follow the instruction VERY carefully
```bash 
irrecord -d /dev/lirc0 ~/lircd.conf
```
>  (Note that in the instructions when it says "click the next button" it does not mean the literal button labeled "Next" on the remote control, it means the next button that you want to be usable with your Raspberry Pi, such as "Vol Up" or "Left.")

* Once all of that is setup, run the following command and start clicking buttons on your remote. If you see output then the remote is configured properly
```bash
irw
```

## LIRC Uses ##

After following all of the steps above, LIRC can now be used in your own programs for whatever your heart desires. If you wish to setup your own streaming service or simply play funny noises at the push of a button then the world is your oyster.

The way to access this new functionality is by having your program open and listen to the AF_UNIX socket. A simple example of this can be found [here](https://github.com/jcconnol/remote-rpi-insulter/blob/fc887670d20597e72a3110403eec43875ea0a682/main.py#L16) or the following code has the extra stuff cut out of it.
```python
SOCKPATH = "/var/run/lirc/lircd"

def init_irw():
    global sock
    sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
    sock.connect(SOCKPATH)

if __name__ == '__main__':
    try:
        init_irw()

    except KeyboardInterrupt:
        if (sock != None):
            sock.close()
```

For the full example from above, if you believe the robot uprising is inevitable and you get tired of saying the same things over and over all day, you can automate that so that at the click of a button you can have your Raspberry Pi say what you command it to with the simple click of a button.

## More Details About LIRC and IR in General ##

If you have read this far then props to you! The rest of this is a further explanation of LIRC and IR and if you want to learn about that kind of thing then feel free to continue.

The most basic element of all of this is infrared, a form of light that has a frequency smaller than that of visible light. 
![light spectrum diagram](/images/spectrum-diagram.png)

We obviously cannot see infrared light with the naked eye but in essence your remote control is just a fancy flashlight that, when a button is clicked, it quickly turns the flashlight on and off in a certain pattern that the receiver can understand.

The whole process looks like the following:
![IR signal from remote to IR receiver to Raspberry Pi](/images/ir-remote.png)

Each command that the remote control puts out has two parts, the device address, to tell the receiver what device is trying to send a command, then the actual command, such as "volume up."
These commands and device addresses are sent in binary to the receiver. One command might be 000 0001 and another might be 000 1001, but the receiver is configured to know what the binary commands are and execute them accordingly. This whole process is a lot like in old movies when people would communicate by shining a flashlight at their far away friend and turn it on and off in morse code. 

## Conclusion ##

Imagine a world where your only form of communication is where people shine a flash light in your eyes and turns it on and off in a particularly annoying and unique pattern that you can interpret as a specific command, such as "take out the trash" and once given this command you are forced to carry out this order. This flashlight communication is, in essence, how IR remote controls. There is a lot more that goes into them but those typically vary between remotes and their different complexities.

Learn about LIRC and how to connect your Raspberry Pi to an IR receiver and control it using a remote

Raspberry Pi 4 Infrared Control (LIRC)

## What is Docker?

Simply said, Docker is pretty much a virtual machine. Don't know what a virtual machine is? Imagine you are a person with too much money and therefore bought a mac. Imagine opening your mac, turning it on normally and on your desktop you could open open an application that is running Windows on your computer. That is a virtual machine. When a computer runs a normal virtual machine it takes a lot of processing power since there is essentially one operating system running inside of another operating system. Docker does this same thing but in a very different way that takes less processing power. This is because Docker doesn't run an entire operating system inside of your normal operating system. Instead Docker reaches into the depths of your machine and uses the low-level mechanics of the host OS to simulate a different environment.

## So What?

Say you are running a program locally and you want to put it on a server but your machine is Windows and the server is running Linux. With Docker, container-based applications can be setup and run independent of what the actual system running the container is. This leads to faster and more consistent development since the application, its dependencies and its configuration are all packaged together, not to mention it also greatly helps solve the "but it works on my machine" problem that arises quite often when debugging.

In simple terms Docker containers can be thought of like a shipping container, it can be carried by ship, truck or train no matter what cargo is actually in the container. 

## Docker Example in 10 Easy Steps

1) Install Node.js if you do not already have it. It can be found [here](https://nodejs.org/en/download/)

2) Install Docker if you have not already. It can be found [here](https://docs.docker.com/engine/install/). These instructions worried me the first time I ran through them but just follow them and read everything and you should be fine.
 
3) Create a new folder for this simple project then cd into that folder using your terminal or command-line shell of your choice.

4) Create a "Dockerfile". This file should have no extension or file type. 

5) In this Dockerfile add the following code:

```bash
FROM node:14
WORKDIR /usr/src/app
COPY package*.json app.js ./
RUN npm install
EXPOSE 3000
CMD ["node", "app.js"]
```

6) Create another file called `index.js` in that same folder

7) Open the `index.js` file in you favorite text editor and add the following to it:

```javascript
const express = require('express')
const app = express()

app.get('/', (req, res) => res.send('Hello World!'))
app.listen(3000, () => console.log('Server ready'))
```

8) Run the following two commands. This should add a `package.json` file and install Node's express package.

```console
npm init -y
npm install express
```

9) Now you can build a docker container with our new application with the following command:

```console
docker build -t examplenode .
```

10) After that takes 10 years to build we are now ready to run the application in a Docker container. This can be done with the following command:

```console
docker run -d -p 3000:3000 --name node-app examplenode
```

11) With that running, you can now go to [http://localhost:3000](http://localhost:3000/) and you can see your application running!

## Considerations For When not to use Docker ##

Docker seems great on the outside and in my experience it works great for a lot of applications but there are a few things that Docker will not do for you.

### Docker Will Not Boost Application Speed ###

Docker containers are lighter weight than a virtual machine but the application still runs on the host's kernel and therefore the application won't be sped up in any way that the computer or server would not normally allow.

In fact, if the proper memory or CPU limits are not in place for a Docker container then the computer might detect that memory is getting low and start shutting off other necessary functions in an attempt to run your program.

### Not Great For Storing Valuable Data ###

Docker files are created inside the container's writeable container layer, therefore if you write data to the container and need to then retrieve it to look at it or move it elsewhere then this will not be an easy task. Even worse, if the container shuts down then that data will be lost forever.

With all these fancy Docker things that few people seem to know about, here is a simple overview of Docker

Docker in Plain English

Known by 100% of all software engineers and coders, "Hello World!" is typically what is implemented to show the first sign of life after successfully implementing or installing a new computer program.

So in the spirit of this being my first blog post, it is only natural to start with "Hello World!"

## History ##

The first version of "Hello World!" was coined in Brian Kernigham’s 1972 book, _A Tutorial Introduction to the Language B_. In this book, the phrase "hi!" is output, because a character constant is limited to 4 ASCII characters. This phrase was later written in [C](https://www.cprogramming.com/) and lengthened to the full "Hello World!" in 1974 by a [Bell Labs](https://www.belllabs.com/) internal memo. After being formally published in the 1978 book _The C Programming Language_, programmers from all over have used it to confirm that their source code, compiler (or JVM, etc.), and configuration are all correct. 

## Implementation ##

"Hello World!" can be output with any programming language, which means that implementations of "Hello World!" can vary greatly. The differences for prevalent programming languages can vary in length and complexity. Take the _Assembly_ example below:

```bash 
 global _main
 extern _GetStdHandle@4
 extern _WriteFile@20
 extern _ExitProcess@4

 section .text
_main:
 ; DWORD bytes; 
 mov ebp, esp
 sub esp, 4

 ; hStdOut = GetstdHandle( STD_OUTPUT_HANDLE)
 push -11
 call _GetStdHandle@4
 mov ebx, eax 

 ; WriteFile( hstdOut, message, length(message), &bytes, 0);
 push 0
 lea eax, [ebp-4]
 push eax
 push (message_end - message)
 push message
 push ebx
 call _WriteFile@20

 ; ExitProcess(0)
 push 0
 call _ExitProcess@4

 ; never here
 hlt
message:
 db 'Hello, World', 10
message_end:
```


Considered a very low level programming language (meaning being closer to being written in 1's and 0's), these 17 lines of Assembly code can be compiled on a Windows machine to output "Hello World!".

On the opposite side of the spectrum, JavaScript is a modern programming language with a lot of modern functionality, such as console logging. This allows someone to output "Hello World!" with just a single line:

```js
console.log("Hello World");
```

This makes modern development and implementation of simple outputs such as "Hello World!" a much simpler task. 

With the perks of modern life, however, come people that just like to watch the world burn, hence the birth of a breed of programming languages that are intentionally made to be difficult. Take [Brainf***](https://en.wikipedia.org/wiki/Brainfuck) , for example. The following is an implementation of "Hello World!":

```brainfuck 
--<-<<+[
 +[
 <+>--->->->-<<<
 ]>
]
<<--.<++++++.<<-..<<.<+.>>.>>.<<<.+++.>>.>>-.<<<+.
```

Although Brainf*** is not a widely used programming language (and definitely not a serious one), it shows that even when people are just messing around, "Hello World!" is still as prevalent as ever. Those simple words could be the start of a new and exciting use of a programming language OR the start of many hours of frustration and combing through StackOverflow.


## Conclusion ##

In the end, "Hello World!" is a very prevalent and relevant tradition in the world of programming, and it does not seem to be going anywhere anytime soon. This is proof that even in this technological age, where everything changes almost all the time, there are still traditions from before the age of floppy disks.

Ever used a new language and started with a 'Hello World!' output? Find out its origin and a couple uses for it.

Blog Posts