Today I want to show you how to start with AWS Auto Scaling Groups. When we need Auto Scaling Groups? It can be a question for a whole, another post. 😀
I have prepared an app on Spring Boot which pull arguments from SQS, calculates prime number bigger than that argument and finally puts the result to the S3 (Simple Storage Service). Everything works quickly on small numbers. When it comes to bigger input (e.g. 10 000 000) it takes really long time. Server is calculating and we are waiting, next number is waiting…
And then… our Scaling Group says – it is a good moment to add some new resources and creates a next server. After all, when there is nothing to calculate, Scaling Group kills them.
Application
First of all, when we want to use AWS in our Maven project we have to add some dependencies.
<!-- AWS SQS CORE -->
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-sqs</artifactId>
<version>1.11.478</version>
</dependency>
<!-- AWS SDK CORE -->
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk</artifactId>
<version>1.11.431</version>
</dependency>
<!-- AWS MESSAGE SQS -->
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>amazon-sqs-java-messaging-lib</artifactId>
<version>1.0.4</version>
</dependency>
<!-- JMS API -->
<dependency>
<groupId>javax.jms</groupId>
<artifactId>javax.jms-api</artifactId>
<version>2.0.1</version>
</dependency>
And plugins to build an apps from maven. They help us later.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<configuration>
<archive>
<manifest>
<mainClass>Application</mainClass>
</manifest>
</archive>
</configuration>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.4.0</version>
<configuration>
<mainClass>Application</mainClass>
</configuration>
</plugin>Now we are going to write our code. There are many ways to provide credentials. Below is one of them (not the best I sure).
BasicAWSCredentials awsCreds = new BasicAWSCredentials(ACCESS_KEY, SECRET_KEY);Then we can connect to our created queue on SQS.
SQSConnectionFactory connectionFactory =
new SQSConnectionFactory(new ProviderConfiguration(),
AmazonSQSClientBuilder.standard()
.withRegion("us-east-2")
.withCredentials(new AWSStaticCredentialsProvider(awsCreds))
.build()
);
SQSConnection connection = connectionFactory.createConnection();I said that we will use S3 to hold the results so we need to define it too.
AmazonS3 s3Client = AmazonS3ClientBuilder.standard()
.withRegion("us-east-2")
.withCredentials(new bAWSStaticCredentialsProvider(awsCreds))
.build();
Finally, it is a good time to create a session to queue and start to receiving the messages.
Session session = connection.createSession(false, Session.CLIENT_ACKNOWLEDGE);
String QUEUE_NAME = "my-example-queue";
MessageConsumer consumer = session.createConsumer(session.createQueue(QUEUE_NAME));
connection.start();
receiveMessages(s3Client, consumer);
connection.close();
System.out.println("Connection closed");Below the receiveMessages method with a basic method which calculates prime number.
private static void receiveMessages(AmazonS3 s3Client, MessageConsumer consumer) {
try {
while (true) {
System.out.println("Waiting for messages");
// Wait 1 minute for a message
Message message = consumer.receive(TimeUnit.DAYS.toMillis(1));
if (message == null) {
System.out.println("Shutting down after 1 minute of silence");
break;
}
if (message.propertyExists("n")) {
int prime = calculatePrimeNumber(2, 0, message.getIntProperty("n"));
System.out.println("Wyliczone: " + prime);
s3Client.putObject("lista-10-odpowiedzi", "primeNumber_" + message.getIntProperty("n"), "For n: " + message.getIntProperty("n") + " prime number is " + prime);
}
message.acknowledge();
System.out.println("Acknowledged message " + message.getJMSMessageID());
}
} catch (JMSException e) {
System.err.println("Error receiving from SQS: " + e.getMessage());
e.printStackTrace();
}
}
private static int calculatePrimeNumber(int number, int i, int n) {
while (true) {
number++;
if (isPrime(number)) ++i;
if (i > n && isPrime(number)) break;
}
return number;
}
static boolean isPrime(int n) {
for (int i = 2; i <= Math.sqrt(n); i++) {
if (n % i == 0)
return false;
}
return true;
}
We can test our program by sending a example message with parameter n.
We can see the result in the S3 backet.
Deploy application on AWS
Ok. If everything looks good we can go through the next steps. Now we are going to deploy our application on AWS. What is more, we have to add our application as a system service. The reason is that, when new servers will be starting, then our app should be started on them. After that it is necessary to create AMI which future servers will be using. We have much work to do so lets go!
I have pushed my code to GitHub. So on the server I just need to install GIT, Maven and Java to compile and run it.
$ cd /etc/systemd/system
$ touch example-app.service
$ vim example-app.service[Unit]
Description = SQS App
After = network.target
[Service]
ExecStart = /home/ubuntu/sqs-example/run.sh
RestartSec = 5
[Install]
WantedBy = multi-user.targetThen go to the project folder and create run.sh file with content as below.
#!/bin/bash
cd /home/ubuntu/sqs-example && mvn exec:javaAnd check if service is starting application correctly.
$ systemctl restart example-app.service
$ systemctl status example-app.service
Now we can create AMI from current instance.
Create Auto Scaling Group
Applications is ready, AMI of instance too. Finally we can focus on creating auto scalling group.
I have created a launch configurations with before created AMI.
Click „Create Auto Scaling Group” button then fill group name, choose network and subnet.
In the next step I set max and min number of instances. What is more there we can set scaling policy. I set when CPU usage 50% then new server should be added.
On list of instances we see our one server from that auto scaling group.
Test it!
Uff… 🙂 Eventually, check how our Scaling Group works. Send via SQS some really big numbers to calculate and wait for a result. 10 000 000, 10 000 001, 10 000 002…
After a few minutes we can see new server with pending status.
And mooooore…
In S3 exists the results of the calculation. After a while servers are automatically terminating.
As you can see it takes time to scale servers, so in our case (calculation) it is not a good solution but it wasn’t be a goal in this post.