In Panoply's S3 data source, you can set the address to point to an S3 bucket or to a specific file or folder. You can also use a prefix or a wildcard character to filter out the particular files or folders you need. The screenshots below show examples of these options:
- Bucket name - Returns all files and folders within a bucket.
Example:
s3://samplebucket-8
- Folder - Returns all files and folders under a folder. Take note of the trailing slash at the end because it makes the filtering specific to a folder. If you don't add it, this option acts similarly to using a prefix (see the Prefix item in this list).
Example:
s3://samplebucket-8/dc/
- Prefix - Returns the files and folders whose name starts with the prefix.
Example:
s3://samplebucket-8/employee
- Wildcard
*
asterisk character - Use it to filter out the files that contain the text you're looking for. This option is valid only for filtering files.
- Example #1 -
s3://samplebucket-8/*employee*
- Example #2 -
s3://samplebucket-8/*.csv
- Example #1 -
- Specific file - Returns a particular file.
Example:
s3://samplebucket-8/dc/villains.jsonl
The options presented above are valid, and each one of them returns a different set of files back. What you decide to choose depends on your use case and the distribution of the files in your S3 bucket.
Comments
0 comments
Please sign in to leave a comment.