Skip to content

Cloud write

The second central method of ecodev-cloud is save_cloud_data, that allows one to save data to the cloud provider specified in the cloud_provider environment variable.

It's interface reads

def save_cloud_data(file_path: Path,
                    data: DATA_TYPE,
                    cloud: Cloud = CLOUD,
                    location: str | None = None
                    ) -> None:

where:

  • file_path: a pathlib Path (by the way, we advise to get rid of all os code when you can. For dealing with files/folders, pathlib is just way simpler and user friendly to use.) specifying where the data will be located in the s3 bucket / blob container. Remember to read the forge key page to learn how to properly... Well, forge a key 😊.
  • data: the data to store on cloud. Read on to find the different types of data that can be saved.
  • cloud (optional): the cloud provider used to connect to a distance storage. Read the installation guide in order to learn how to connect to your S3 like/blob cloud provider, and also consult cloud details. By default, the environment variable cloud_provider is used.
  • location (optional): the s3 bucket/blob container on which to connect. By default the s3_bucket_name environment variable is used if cloud=Cloud.AWS, and container is used if cloud=Cloud.Azure.

As of 2024/05, this method can write the following file types (it relies on the file extension given in file_type to use the appropriate saver):

  • csv
  • xlsx
  • npy
  • npy.npz (compressed numpy)
  • json
  • tex
  • txt
  • png
  • zip
  • shp (subtlety: the shapefile will be stored zipped in order to easily retrieve it. You can go inspect the source code, and the save_shp method to learn more)