Questions tagged [amazon-s3]
Amazon S3 (simple storage service) is an online object storage service from Amazon Web Services. QUESTIONS MUST BE ABOUT PROGRAMMING. Questions about general S3 support, functionality, configuration, etc. are OFF-TOPIC.
amazon-s3
50,880
questions
0
votes
0
answers
21
views
How do i get my Python .env credentials file into Ubuntu EC2?
I have a .env file which my app.py file references, containing my AWS credentials. How do i get this into my AWS Ubuntu EC2 machine?
I can't even upload it to GitHub.
If there's any security issues ...
- 415
0
votes
0
answers
5
views
From AWS Cloudshell , Getting error while push zip file to S3 Bucket
I am using AWS Cloudshell to create .zip file and push it to AWS S3 Buckets. Few Weeks ago, from cloudhshell to S3 Bucket it was working fine.
Today I got the error for running same aws s3 command in ...
- 970
-1
votes
0
answers
15
views
Django hosted on Render - Media Files/template vars
I've got a Django app that stores its static and media files in an AWS S3 bucket. Retrieval, usage and storage of these files work fine locally. The issue is, when accessing through the live site ...
- 35
0
votes
0
answers
13
views
Elasticsearch’s S3 Snapshot Repository: The Mystery of the Inaccessible S3 Bucket ==> Access Denied
Elasticsearch Version
7.17.12
Installed Plugins
repository-s3
Java Version
JAVA_RUNTIME_VERSION="20.0.2+9-78
OS Version
Linux elasticsearch-data-0 5.10.214-202.855.amzn2.x86_64 #1 SMP Tue Apr 9 ...
0
votes
0
answers
18
views
Why is my destination prefix given [Object Object] as its output in AWS-SDK?
I am writing this code where im using aws sdk to copy files from one place to another dir, here destinationprefix is supposed to handle to destination directory, there the variable has to be put, idk ...
1
vote
0
answers
34
views
How can invoke a Lambda function with AWS Signature auth without using Postman?
I have a simple Lambda function in Python that converts a text string into speech and saves it to an S3 bucket. I want to then get that mp3 file (or URL) as a response.
I will invoke the Lambda ...
- 19
1
vote
1
answer
15
views
Getting 'Could not load credentials from any providers' error while using AWS-SDK in node.js
I am writing a code to copy a file from one to another and here is the code
Here i am using the code to first find if a file exists and if doesnt it makes one by copying files from one to the other
...
-1
votes
0
answers
7
views
Automating the Download of All Images from Retool Storage to Amazon S3
I am currently working on a project where I need to download all images stored in Retool and transfer them to Amazon S3. I am looking for guidance or best practices to automate this process ...
- 1
0
votes
0
answers
27
views
I keep getting an access denied error on my lambda function
I'm a beginner and I keep getting this error message when testing a Lambda function. I'm trying to do a simple write file to an S3 directory bucket.
Here is the PHP code:
import json
import boto3
...
- 19
-2
votes
2
answers
29
views
How to check if a file uploaded to AWS S3 has the same content as a local file
We believe that the S3 checksum is useless for huge files larger than 1GB, as it is a further hashed value of chunks separated by an arbitrary number of bytes.
There is a 1 GB file uploaded to AWS S3.
...
- 399
0
votes
1
answer
22
views
Debounced s3 notifications to execute a lambda
I want to run a task once whenever a specific folder is modified in a bucket, no matter how many files are in the modification, it has to run once in that period of time.
The issue with the classic s3 ...
- 61
1
vote
0
answers
15
views
How to send File Object from client component to server actions in Next.js 14?
I have been trying to upload the image/video to AWS S3 storage via pre-signed URL. After I get the URL from the backend I am sending the URL and File component to the server actions to perform put ...
- 109
0
votes
0
answers
10
views
Is it possible to use Background Fetch Api for uploading file..?
I'm working on implementing the Background Fetch API in my Vue project to enable file uploads to an S3 bucket. The goal is to allow the upload process to continue even if the user closes the screen, ...
- 11
0
votes
0
answers
25
views
Laravel Vapor gives error : - The variables may not be greater than 2000 characters
I downloaded the env file from staging env using vapor env:pull staging and I updated it with my local env variables and then I run vapor env:push and then it gives me an error like:
The variables ...
- 1
0
votes
0
answers
14
views
AWS S3 form-based POST file upload ignores Content-Type
When using a form POST to upload a file to an S3 bucket, S3 ignores the Content-Type/MIME type of the file.
The file is uploaded successfully. The file form data field contains a Blob with a Content-...
- 163k
0
votes
1
answer
24
views
Add A Single Column CSV To DynamoDb Table Via S3
I have a large CSV file of 265,000,000 domain names, one domain per line. I want to upload this file to AWS S3 and then upload the file to DynamoDb. I have some questions.
Can I make the domain on ...
- 145
0
votes
0
answers
19
views
Zip multiple S3 files and stream the archive back to S3 using limited memory
I am trying to use .NET 8 to zip a large amount of files stored in S3 and save the archive in another S3 location, while consuming a limited amount of memory.
Basically my needs are:
I have a bunch ...
- 3,792
-1
votes
0
answers
14
views
Efficient Handling of Media Uploads and Processing via EC2 and S3
I am developing a mobile application that needs to handle media uploads. The current design is as follows:
Upload to S3: The mobile client directly uploads the media file to an S3 bucket using a PUT ...
- 423
0
votes
0
answers
19
views
Aws sdk s3 to s3 copyObject checksum is null in CopyObjectResponse
When doing a s3 to s3 copy using aws sdk, I want to save the returned checksum from CopyObjectResponse, so that I can use it later to verify if file is modified or not
but checksum value returned is ...
0
votes
0
answers
18
views
Get File uploaded to AWS S3 bucket and read into Pillow to convert it
I have a Lambda Function which triggers when a file is uploaded to it's bucket. I want to
open said file with Pillow and convert it tp JPEG.
Code looks like this:
import os
import json
import boto3
...
- 95
0
votes
0
answers
15
views
AWS Athena Error: Modifying Hive table rows is only supported for transactional tables
I am not able to perform delete operation on row in AWS Athena tables. It is throwing below error as:
NOT_SUPPORTED: Modifying Hive table rows is only supported for transactional tables
This query ran ...
- 594
0
votes
1
answer
41
views
Upload video to S3 from Android using Nextjs Backend
This is a very straightforward and simple question. I am writing an API's backend for android app. As usual the android app will call API's that i will deploy on my backend one will be Upload to S3 ...
- 25
0
votes
0
answers
42
views
Unable to Access an S3 Bucket From R
I have an AWS buckets with several shape files and I'm trying to connect to that bucket from R
Set environment variables
library(aws.s3)
Sys.setenv(
"AWS_ACCESS_KEY_ID" = "",
...
- 1,458
-1
votes
0
answers
19
views
AWS Lamda event getting triggered multiple times
I am implementing a solution where whenever a new file is placed in S3 I am triggering an event in my lambda function which inserts the data further to AWS Athena.
But the issue is event is getting ...
- 594
0
votes
2
answers
27
views
Denied access to s3 URL
I have a node function where I'm attaching a pdf from a s3 URL and I keep getting a 403 error. This happens in the program as well as the browser. Here is the Nodemailer function I'm trying to attach ...
- 1,099
0
votes
0
answers
14
views
Getting CORS error in AWS (Cloudfront distribution + S3 setup)
Earlier I was accessing this cloudfront distribution from one repo, localhost:4000, and from stage env. It was all working fine. I set up a new localhost localhost:4800 and tried accessing the same ...
- 266
1
vote
0
answers
38
views
Getting "cannot schedule new futures after interpreter shutdown" error while using boto3.client download_file and upload_file functions
I am using boto3.client.download_file() and boto3.client.upload_file() functions, For python3.8 environment it's working fine, but after upgrading to python3.10, getting "RuntimeError: cannot ...
- 37
-1
votes
1
answer
27
views
AWS: Bucket URL works but not the A-record
I have an S3 bucket configured as a static web site with public access and the standard permission policy. It's hosted in US East (region 1). The URL for this bucket works.
I've created an A-record ...
- 1
1
vote
0
answers
23
views
Error parsing CSV File when copying Data to Snowflake after July 3rd Incident
I am encountering an error while trying to copy a CSV file into Snowflake from an S3 bucket. This process was functioning correctly until an incident occurred in Snowflake on July 3rd. The error ...
- 43
0
votes
0
answers
17
views
AWS crawler creating Null values for partion columns
I am having some country level partitioned data in s3 and crawler is crawling the this root folder and creating a table. No Null value is there for country code. But when looked in the Athena, there ...
- 41
1
vote
1
answer
31
views
Saving PDF from S3 .pipe is not a function
I'm ultimately trying to attach a file from s3 without saving it to disc, but I'm having trouble getting the file in the first place.
Here is my code:
const s3 = require('@aws-sdk/client-s3')
...
- 1,099
0
votes
0
answers
38
views
Random cURL error 35 when call to S3-compatible API
Here is my situation :
I'm trying to use a S3-compatible API, called from a PHP script. (Ionos : documentation here)
I want to do some API requests to get files from the S3 and buckets informations.
...
- 1
0
votes
0
answers
17
views
S3 throws 403 when I restrict role to vpce policy using principal
I wanted to restrict my VPC Endpoint policy with by specifying my IAM role in principal
doc which I followed: https://docs.aws.amazon.com/vpc/latest/privatelink/vpc-endpoints-access.html
{
"...
- 57
1
vote
0
answers
26
views
AWS VPC: Lambda functions experiencing slow performance and timeouts
I recently modified my VPC configuration, and now I'm experiencing significant performance issues with my Lambda functions. Here's my current setup:
VPC Configuration:
4 subnets total:
3 subnets (eu-...
- 172
1
vote
0
answers
32
views
AWS SDK object is taking longer time than expected and queueing up calls
I have an API in NodeJS server hosted in AWS EKS fargate which performs some operations(some related to AWS KMS) and then upload a BSON to S3. I have configured a load balancer timeout of 5 minutes.
...
1
vote
0
answers
57
views
Performance Degradation in Aws Spot Instances while running the same process on multiple cores
Problem : I am observing a performance degradation of over 100 % when I am running the same program on all the cores of Aws spot instance by pinning them on separate cores in comparison to running a ...
0
votes
0
answers
24
views
How can i concatenate (dropzone.js) front chunk file with laravel controller
@section('css')
<link href="https://cdnjs.cloudflare.com/ajax/libs/dropzone/5.9.3/min/dropzone.min.css" rel="stylesheet">
@endsection
@section('content')
<h1>File ...
-3
votes
1
answer
25
views
S3 cross account (different tenants) (same region) replication, do buckets need to be publicly accessible? [closed]
We need to setup S3 bucket replication from a SaaS provider (in a different tenant and account, but same S3 region) to our S3 bucket to capture logs from the SaaS application.
We have a bucket policy ...
0
votes
0
answers
21
views
My vite web app not opening on linkendIn and instagram mobile browser
I have a vite app, and i'm trying to promote it on instagram and have attached my link to it.
But the link is not opening in the instagram app inbuilt browser and same for linkendIn only in ios ...
0
votes
0
answers
21
views
Troubleshooting Kafka Integration with Spark Streaming on Amazon EMR Serverless
Objective:
To set up a streaming job on Amazon EMR Serverless to process weather data from Amazon MSK (Managed Streaming for Apache Kafka) and write the word count results to an S3 bucket.
Steps Taken:...
0
votes
0
answers
32
views
Error in getting data from EventBridgeEvent
I have step function with lambda_1 as starting point.
This step function is going to be triggered when object is added to S3. I am getting the event and trying to fetch objectKey and bucket from that ...
- 1,109
0
votes
0
answers
31
views
Streaming data from delegate to function call
I have a 3rd party library that calls a delegate every time there is a buffer of data, and I need to get this buffer into a stream and send the stream to S3.
So I use a FileStream to write to a ...
- 97
0
votes
0
answers
8
views
Page show blank white screen when deploy new code
I had a trouble with my site, I am using vue3 and amplify to deploy my site. My problem is when I deploy to new code, when deploying is finish and I visit my site again, my screen show blank white ...
- 400
0
votes
0
answers
11
views
Assign Bucket Permissions to Users Who Created That Bucket
I created a user and policy which allows an IAM user to create an s3 Bucket. I also want that user to have full permissions on the bucket that they create but not on any other existing bucket. How ...
0
votes
0
answers
9
views
Using Kotlin and AWS SDK, copy file without file without triggering AWS S3 event
Using Kotlin and AWS SDK, how to copy a file in a s3 bucket without triggering the event notification? or not triggering the lambdas connected to the event notification?
https://docs.aws.amazon.com/...
- 1,289
0
votes
0
answers
9
views
Disable secrets manager and s3 for integration testing when contextLoads is called
I have a SpringBoot application which I am trying to test. I'm struggling with this test
@SpringBootTest
@ActiveProfiles("test")
public class LicenceApplicationTests
{
@Test
@...
- 1
0
votes
0
answers
26
views
Data is getting truncated when csv is uploaded to s3
I am facing an issue in aws s3 csv upload. The files i am uploading is not that huge has max size of 10mb or less atleast for now
The upload is success but some data is getting truncated( very few ...
0
votes
1
answer
29
views
Nuxt.js application generating only index.html
I am facing an issue with my Nuxt.js application, which is deployed on an S3 bucket. Despite the configuration, only the index.html page is being generated, and any route other than the root (/) ...
-1
votes
1
answer
25
views
Get Object from Amazon S3 without exception
I'm using C# SDK for .NET Core to fetch objects from the store with GetObjectAsync. However, it might be that there is no such object that the program requests. In this case, GetObjectAsync throws ...
- 1,629
0
votes
1
answer
30
views
How to Run AWS lambda code parallel with SNS to SQS architecture
I have a problem. Let's say there's an external system, and an AWS Lambda function retrieves data from this system, converts it to a CSV, stores it on S3, and triggers an SNS event. This event then ...
