Collect Logs for AWS Fargate

You can use the Moogsoft Onprem Webhook integration to enable log collection for AWS Fargate using AWS FireLens.

For information on how the AWS Fargate compute engine for Amazon ECS lets you run containers without server or cluster management, see AWS Fargate. For information on routing AWS Fargate container logs to AWS FireLens, see Using Firelens.

Log collection

The following diagram illustrates how FireLens sends container logs from AWS Fargate to Moogsoft Onprem:

You can use either Fluentd or Fluent Bit output in your ECS Task Definition for Firelens, but Fluent Bit is a good choice because it uses significantly fewer resources than Fluentd.

Before you begin

Before you can ingest logs into Moogsoft Onprem from FireLens:

If you haven't already, install and configure the AWS CLI and ECS CLI.
Set up a Webhook Integration in Moogsoft Onprem. Make note of the full URL.

Create an AWS task definition and service

You must create a task definition to run Docker containers in Amazon ECS. You can define multiple containers in a task definition. The parameters that you use for the task definition depend on the launch type for the task. For more information about available parameters and the launch types they are valid for in a task definition, see ECS Task Definition Parameters.

Create the launch type that best suits your environment:

Fargate - ECS manages your clusters.
EC2 - you manage your clusters.

Create a Fargate launch type

To create a new task definition with a log router container and application container using JSON:

Create a new ECS Task Iam Role.
Create a JSON file with your task definition specifies the awsfirelens log driver and that directs the logs to Moogsoft Onprem. For example, a file called fargate_moogsoft.json:
```
"logConfiguration": {
	"logDriver": "awsfirelens",
	"options": {
		"Name": "http",
		"Host": "milliways.moogsoft.io",  // this is an example FQDN, replace with your actual
		"URI": "/data", // this is an example PATH, replace with your endpoint
		"Port": "443",
		"tls": "on",
		"tls.verify": "on",
		"Format": "json_lines"
	}
}
```
The sample JSON creates an httpd application container. Replace httpd for the falue of image with the application you want to deploy. For example:
```
"essential": true,        "image": "your_application",        "name": "app",                "logConfiguration": {
```
The sample JSON configures the FireLens Log Router to emit its own logs to the CloudWatch group "awslogs-ecs-fargate-moog”.You should send the FireLens logs into CloudWatch to debug Fluent Bit Issues.
If you want to keep this configuration, create a log group named awslogs-ecs-fargate-moog in us-west-1. Also replace the task execution role if it is named other than the default executionRoleArn and populate the account id. For example:
```
"executionRoleArn": "arn:aws:iam::<account_id>:role/ecsTaskExecutionRole"
```
You can also create a log group with a different name. If you do, update the task definition JSON with the new CloudWatch group name. For example:
```
"logConfiguration": {
             "logDriver": "awslogs",
             "options": {
                 "awslogs-group": "awslogs-ecs-fargate-moog",
                 "awslogs-region": "us-west-1",
                 "awslogs-stream-prefix": "awslogs-ecs-fargate-moog"
             }
        }
```

Create the task definition. For example:

aws ecs register-task-definition --region us-west-1 --cli-input-json  file:///fargate_moogsoft.json

Launch the service. For example:

aws ecs create-service --cluster fargate-ecs-firelens-cluster --service-name firelens-moogsoft-fargate --task-definition firelens-moogsoft-fargate:1 --desired-count 1 --region us-west-1 --launch-t--network-configuration "awsvpcConfiguration={subnets=[subnet-1,subnet-2],securityGroups=[sg-1],assignPublicIp=ENABLED}"

The command requires subnets and the security groups for the service. Substitute the cluster VPC, subnet, and security group variables with real values for your environment. You can determine your cluster VPC, subnets, and associated security group from the AWS console.

You can use the following command to determine the security group for the VPC ID:

aws ec2 describe-security-groups --filters Name=vpc-id,Values=VPC_ID --region us-east-2

Create an EC2 lauch type

To create an EC2 launch type and configure collection for EC2 container logs:

Create a JSON file with your task definition specifies the awsfirelens log driver and that directs the logs to Moogsoft Onprem. For example, a file called ec2.moogsoft.json:

"logConfiguration": {
	"logDriver": "awsfirelens",
	"options": {
		"Name": "http",
		"Host": "milliways.moogsoft.io",  // this is an example FQDN, replace with your actual
		"URI": "/", // this is an example PATH, replace with the endpoint from the webhook
		"Port": "443",
		"tls": "on",
		"tls.verify": "on",
		"Format": "json_lines"
	}
}

Create a task definition and service for EC2. For example

aws ecs register-task-definition --region us-west-1 --cli-input-json  file:///ec2_moogsoft.json