Scheduling Fargate tasks to start at scheduled time of day and stop at the end of day

Hi

We have changed from using ECS clusters running EC2 services and tasks to using ECS clusters running services with Fargate tasks. Each of these ECS clusters represents a group of tasks making up an instance of an application.

We need to find a way to easily schedule the scaling of the tasks within a service based on time, e.g. to shut down and restart our test and QA environments overnight and at weekends.

We did consider Kubernetes (EKS) but decided to go the ECS route due to pricing and deployment effort considerations. I’m sure that some other smaller companies may come to the same conclusion.

We raised a ticket with AWS and received the following response -

I understand that you would like to know how to schedule fargate tasks to run on scheduled time of the day and stop at the end of the day as a cost optimization initiative for your development and test environments.

Currently, ECS supports only scheduling starting of tasks using cron like expressions or cloudwatch events for the fargate launch type as here [1]. However natively this primarily is meant for short running tasks and does not have a native option to stop tasks on a schedule. In other words, this is for tasks that are only meant to perform a specific action and then stop, and one would consider having your containers simply exit once the operation completes. This way you would not need to manually stop the tasks.

Therefore for your use case you would use the ECS schedule tasks cron to start your task in the morning, however, to stop the task the work around, would be to consider use of a lambda function that would get triggered by a CloudWatch event to update the desired count of tasks in your service to 0 at night. Here are documentation that would help guide you in this case [2][3], please test adjusting the code to stop task.

I would also like to mention that there is an ongoing feature request with our service team however we currently do not have an ETA on when this will be released and I have added this case as part of our customer requirements.

I trust this correspondence proves useful. However Should you have any further issues or need additional clarification, please feel free to get back to us and we will be happy to assist you.

References

[1] Scheduled tasks

https://docs.aws.amazon.com/AmazonECS/latest/developerguide/scheduled_tasks.html

[2] Tutorial: Listening for Amazon ECS CloudWatch Events - https://docs.aws.amazon.com/AmazonECS/latest/userguide/ecs_cwet.html

[3] https://github.com/s7anley/aws-ecs-service-stop-lambdaemphasized text**

Any help anyone could provide would be greatly appreciated.

Hi Karl,

thanks for posting your question in the cloudonaut community.

What kind of services are you running on ECS? Batch processing, request-response (e.g., web server), …?

One option could be to create ECS services and use a Lambda function to increase/decrease the number of running tasks based on a schedule (CloudWatch event).

Hi Andreas

Thanks for your prompt response.

We are running both production and development web servers on AWS. We were using CloudRanger to auto scale the EC2 tasks, but since switching to Fargate, this is not supported by them.

Is using Lambda functions and Cloudwatch the simplest way to achieve this with Fargate?

Karl

image628792.png

image383890.png

image365557.png

image549393.png

image672063.png

Ok, so you should definitely use ECS services to run your tasks (containers). A Lambda function triggered by a CloudWatch Event rule (schedule), allows you to update the desiredCount automatically. For example, set the desiredCount to 2 at 07:00 am and reduce it to 0 at 08:00 pm.

Check out https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_UpdateService.html to learn more.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.