Skip to content
Snippets Groups Projects
s3cmd.md 1.76 KiB
Newer Older
  • Learn to ignore specific revisions
  • Jan Siwiec's avatar
    S3
    Jan Siwiec committed
    # s3cmd
    
    S3cmd is a command-line tool for managing data in Amazon Simple Storage Service (S3). It is a free and open-source tool that is available for Linux, macOS, and Windows.
    
    Jan Siwiec's avatar
    Jan Siwiec committed
    
    For more information, see [https://docs.e-infra.cz/storage/object-storage/s3cmd/][a].
    
    Jan Siwiec's avatar
    S3
    Jan Siwiec committed
    
    We recommend using the S3cmd tool already installed on Karolina.
    
    
    Jan Siwiec's avatar
    Jan Siwiec committed
    ## How to Load Modules:
    
    Jan Siwiec's avatar
    S3
    Jan Siwiec committed
    
    To see loaded modules, use:
    
    ```
    ml
    ```
    
    To load modules, use:
    
    ```
    ml s3cmd
    ```
    
    To see module info, use:
    
    ```
    ml spider s3cmd
    
    Jan Siwiec's avatar
    Jan Siwiec committed
    ```
    
    Jan Siwiec's avatar
    S3
    Jan Siwiec committed
    
    
    Jan Siwiec's avatar
    Jan Siwiec committed
    For more information on how to work with modules, see
    
    Jan Siwiec's avatar
    S3
    Jan Siwiec committed
    [https://docs.it4i.cz/software/modules/lmod/][1].
    
    ## Configuration
    
    The s3cmd configuration file is named `.s3cfg` and it is located in the user's home directory, e.g. `/home/username/($HOME)`.
    
    Jan Siwiec's avatar
    Jan Siwiec committed
    
    
    Jan Siwiec's avatar
    S3
    Jan Siwiec committed
    The configuration file has to be created first:
    
    ```
    
    Jan Siwiec's avatar
    Jan Siwiec committed
    s3cmd --configure
    
    Jan Siwiec's avatar
    S3
    Jan Siwiec committed
    ```
    
    For configuration steps, follow the [e-INFRA CZ guide][b].
    
    For the list of basic S3cmd commands, see [https://docs.e-infra.cz/storage/object-storage/s3cmd/#basic-s3cmd-commands][c].
    
    ## FAQ
    
    Error when creating bucket:
    
    ```
    ERROR: S3 error: 400 (InvalidLocationConstraint): The specified location-constraint is not valid
    ```
    
    Jan Siwiec's avatar
    Jan Siwiec committed
    
    
    Jan Siwiec's avatar
    S3
    Jan Siwiec committed
    Solution: set location:
    
    ```
    s3cmd --bucket-location=":default-placement" mb s3://newbucketname
    ```
    
    ## Useful Links
    
    * [Introduction][d]
    * [Usage][e]
    * [Common S3cmd Errors][g]
    * [Configuration erros][f]
    
    [1]: https://docs.it4i.cz/software/modules/lmod/
    
    [a]: https://docs.e-infra.cz/storage/object-storage/s3cmd/
    [b]: https://docs.e-infra.cz/storage/object-storage/s3cmd/#configuration-of-s3cmd-tool
    [c]: https://docs.e-infra.cz/storage/object-storage/s3cmd/#basic-s3cmd-commands
    [d]: https://s3tools.org/s3cmd
    [e]: https://s3tools.org/usage
    [f]: https://docs.aws.amazon.com/AmazonS3/latest/API/ErrorResponses.html
    [g]: https://s3tools.org/kb/category1.htm