A cloud-based object storage service, Amazon S3 (Simple Storage Service) allows businesses and individuals to store their data securely within buckets. They can store and protect any amount of data such as data lakes, websites, mobile applications, backup and restore, archive, enterprise applications, IoT devices, and big data analyticsinto the bucket.
A bucket is a container for objects.
Its easy to use features like interface and management helps you optimize and organize the data according to business-specific requirements.
Features of AWS S3
Store data with changing or unknown access patterns with the S3 Intelligent-Tiering feature.
Various storage managenment features used manage costs, meet regulatory requirements, reduce latency, and save multiple distinct copies of your data for compliance requirements.
Amazon S3 provides features for auditing and managing access to your buckets and objects.
You can refer to the to know more about the features.
Getting started
The AWS S3 destination is a secure method of connecting to your S3 buckets. Lemnisk uses it's own IAM Roles to access your S3 buckets.
Connection
Connection
Web
Mobile
​Device Mode
🚫
🚫
Cloud Mode
✅
✅
Configuration
In order to set up Amazon S3 as a destination in Lemnisk, you will have to login to your console and then (you can also choose an existing one, if any).
Irrespective of the region, the raw logs of data from AWS S3, received by Lemnisk are put into your S3 bucket.
The data is processed from your sources and events are collected in batches by Lemnisk. Once every hour, a process is initiated where the batches are uploaded to a Lemnisk S3 bucket, and then securely copied to your own S3 bucket. However, there are no limitations to the data size.
For Lemnisk to access your S3 bucket, you will have to add the below bucket policy for a smooth transition of data.
You will have to perform the following steps in the Lemnisk app to add AWS S3 as a destination.
Navigate to CONNECT > Connections > Destinations and click +Create Destination.
Step 1: A Catalog pop-up appears. Select Amazon S3 and click Continue2
Step 2: Select a source of your choice by checking the box adjacent to the source and then click Continue.
Step 3: Type a unique name that acts as your destination identifier. Click Create Destination.
Step 4: The Overview tab appears with the details of the destination you just created.
Step 5: Toggle the Status button to activate/ deactivate the destination.
Step 6: Click +Add Sources and select a source(s) of your choice by checking the box adjacent to the source and then click Continue.
Step 7: Type a unique name that acts as your bucket identifier. Click Update Destination.
You can add a custom prefix path before typing the bucket name in the Bucket Name text box. For example, if AbcdS3 is your bucket name, and AWS is the prefix. The bucket name along with the prefix would be AWS AbcdS3.
FAQs
​Q1. What connection does the AWS S3 work on?
The AWS S3 is available in Cloud mode for both web and mobile.
Q2. How do I configure AWS S3?
Q3. How is the data processed in Lemnisk?
The data is processed from your sources and events are collected in batches by Lemnisk. Once every hour, a process is initiated where the batches are uploaded to a Lemnisk S3 bucket, and then securely copied to your own S3 bucket. However, there are no limitations to the data size.
Q4. How do I set up AWS S3 as a destination in Lemnisk?
You will have to perform the following steps in the Lemnisk app to add AWS S3 as a destination.
Navigate to CONNECT > Connections > Destinations and click +Create Destination.
A Catalog pop-up appears. Select Amazon S3 and click Continue.
Select a source and then click Continue.
Type a unique name and click Create Destination.
The Overview tab appears with the details of the destination you just created.
Toggle the Status button to activate/ deactivate the destination.
Click +Add Sources. Select a source(s) of your choice and then click Continue.
Type a unique name and click Update Destination.
Q5. How to add a prefix path to the destination name?
You can add a custom prefix path before typing the bucket name in the Bucket Name text box. For example, if AbcdS3 is your bucket name, and AWS is the prefix. The bucket name along with the prefix would be AWS AbcdS3.
You will have to log in to your console and then (you can also choose an existing one, if any).
