s3.bucket.name
The S3 Bucket.
- Type: string
- Importance: high
s3.object.tagging
Tag S3 objects with start and end offsets, as well as record count.
- Type: boolean
- Default: false
- Importance: low
s3.region
The AWS region to be used the connector.
- Type: string
- Default: us-west-2
- Valid Values: [us-iso-east-1, us-isob-east-1, us-gov-east-1, us-gov-west-1, ap-east-1, ap-northeast-1, ap-northeast-2, ap-south-1, ap-southeast-1, ap-southeast-2, ca-central-1, eu-central-1, eu-north-1, eu-west-1, eu-west-2, eu-west-3, me-south-1, sa-east-1, us-east-1, us-east-2, us-west-1, us-west-2, cn-north-1, cn-northwest-1]
- Importance: medium
s3.part.size
The Part Size in S3 Multi-part Uploads.
- Type: int
- Default: 26214400
- Valid Values: [5242880,…,2147483647]
- Importance: high
s3.credentials.provider.class
Credentials provider or provider chain to use for authentication to AWS. By default the connector uses DefaultAWSCredentialsProviderChain
.
- Type: class
- Default: com.amazonaws.auth.DefaultAWSCredentialsProviderChain
- Valid Values: Any class implementing: interface com.amazonaws.auth.AWSCredentialsProvider
- Importance: low
aws.access.key.id
The AWS access key ID used to authenticate personal AWS credentials such as IAM credentials. Use only if you do not wish to authenticate by using a credentials provider class via s3.credentials.provider.class
.
- Type: string
- Default: “”
- Importance: high
aws.secret.access.key
The secret access key used to authenticate personal AWS credentials such as IAM credentials. Use only if you do not wish to authenticate by using a credentials provider class via s3.credentials.provider.class
.
- Type: password
- Default: [hidden]
- Importance: high
s3.ssea.name
The S3 Server Side Encryption Algorithm.
- Type: string
- Default: “”
- Valid Values: [, AES256, aws:kms]
- Importance: low
s3.sse.customer.key
The S3 Server Side Encryption Customer-Provided Key (SSE-C).
- Type: password
- Default: [hidden]
- Importance: low
s3.sse.kms.key.id
The name of the AWS Key Management Service (AWS-KMS) key to be used for server side encryption of the S3 objects. No encryption is used when no key is provided, but it is enabled when aws:kms
is specified as encryption algorithm with a valid key name.
- Type: string
- Default: “”
- Importance: low
s3.acl.canned
An S3 canned ACL header value to apply when writing objects.
- Type: string
- Default: null
- Valid Values: [private, public-read, public-read-write, authenticated-read, log-delivery-write, bucket-owner-read, bucket-owner-full-control, aws-exec-read]
- Importance: low
s3.wan.mode
Use S3 accelerated endpoint.
- Type: boolean
- Default: false
- Importance: medium
s3.compression.type
Compression type for files written to S3. Applied when using JsonFormat or ByteArrayFormat. Available values: none, gzip.
- Type: string
- Default: none
- Valid Values: [none, gzip]
- Importance: low
s3.compression.level
Compression level for files written to S3. Applied when using JsonFormat or ByteArrayFormat.
- Type: int
- Default: -1
- Valid Values: -1 for system default, or [-1,…,9] for levels between no compression and best compression
- Importance: low
s3.part.retries
Maximum number of retry attempts for failed requests. Zero means no retries. The actual number of attempts is determined by the S3 client based on multiple factors including, but not limited to: the value of this parameter, type of exception occurred, and throttling settings of the underlying S3 client.
- Type: int
- Default: 3
- Valid Values: [0,…]
- Importance: medium
s3.retry.backoff.ms
How long to wait in milliseconds before attempting the first retry of a failed S3 request. Upon a failure, this connector may wait up to twice as long as the previous wait, up to the maximum number of retries. This avoids retrying in a tight loop under failure scenarios.
- Type: long
- Default: 200
- Valid Values: [0,…]
- Importance: low
format.bytearray.extension
Output file extension for ByteArrayFormat. Defaults to .bin
.
- Type: string
- Default: .bin
- Importance: low
format.bytearray.separator
String inserted between records for ByteArrayFormat. Defaults to System.lineSeparator()
and may contain escape sequences like \n
. An input record that contains the line separator will look like multiple records in the output S3 object.
- Type: string
- Default: null
- Importance: low
s3.proxy.url
S3 Proxy settings encoded in URL syntax. This property is meant to be used only if you need to access S3 through a proxy.
- Type: string
- Default: “”
- Importance: low
s3.proxy.user
S3 Proxy User. This property is meant to be used only if you need to access S3 through a proxy. Using s3.proxy.user
instead of embedding the username and password in s3.proxy.url
allows the password to be hidden in the logs.
- Type: string
- Default: null
- Importance: low
s3.proxy.password
S3 Proxy Password. This property is meant to be used only if you need to access S3 through a proxy. Using s3.proxy.password
instead of embedding the username and password in s3.proxy.url
allows the password to be hidden in the logs.
- Type: password
- Default: [hidden]
- Importance: low
s3.http.send.expect.continue
Enable or disable use of the HTTP/1.1 handshake using EXPECT: 100-CONTINUE during multi-part upload. If true, the client will wait for a 100 (CONTINUE) response before sending the request body. Else, the client uploads the entire request body without checking if the server is willing to accept the request.
- Type: boolean
- Default: true
- Importance: low
behavior.on.null.values
How to handle records with a null value (for example, Kafka tombstone records). Valid options are ignore
and fail
.
- Type: string
- Default: fail
- Valid Values: [ignore, fail]
- Importance: low
sts.role.arn
Role ARN to use when starting a session.
- Type: string
- Default: null
- Importance: high
sts.role.external.id
The role external ID used when retrieving session credentials under an assumed role.
- Type: string
- Default: null
- Importance: medium
sts.role.session.name
Role session name to use when starting a session.
- Type: string
- Default: null
- Importance: high