Guides

Cloud Bucket Configuration

Suggest Edits

Overview

This document outlines the requirements and steps needed to allow Scope3 to synchronize data from your cloud provider to Scope3 integrations. Depending on your cloud provider, the setup will be different.


Google Cloud Storage

What you will need to get started:

Service Account: 

project-449115859788@storage-transfer-service.iam.gserviceaccount.com

Roles:

Storage Object Viewer (roles/storage.objectViewer)
Storage Legacy Bucket Reader (roles/storage.legacyBucketReader)

Follow the below steps to configure access to a Google Cloud Storage bucket:

  • Go to the Cloud Storage Buckets page in the Google Cloud console.
  • Click the Bucket overflow menu button associated with the bucket to which you want to grant a principal a role.
  • Choose Edit access and click the + Add principal button.
  • In the New principals field, enter the Scope3 transfer service account email: project-449115859788@storage-transfer-service.iam.gserviceaccount.com.
  • Select Storage Object Viewer from the Select a role drop-down menu.
  • Click the Add another role button and select Storage Legacy Bucket Reader.
  • Click the Save button.

Items to send to Scope3:

  • GCS bucket URI e.g gs://example-bucket-name
  • Optional: a sub-folder in the bucket all items will be stored in if different from the root directory
  • Optional: A date on which to only synchronize files that are created/modified after

Write permissions (optional)

As well as reports being available in the CSP, data can be returned to your cloud bucket. Please add the following permissions to your bucket to make this possible:

  • In the Google Cloud console, go to the Cloud Storage Buckets page.
  • Click the Bucket overflow menu button associated with the bucket to which you want to grant a principal a role.
  • Choose Edit access and click the + Add principal button.
  • In the New principals field, enter the Scope3 service account email: [email protected].
  • Select Storage Object Creator Writer from the Select a role drop-down menu.
  • Select Storage Object Viewer Writer from the Select a role drop-down menu.
  • Click the Save button.

Amazon S3

What you will need to get started:

Supported AWS regions:

af-south-1
ap-east-1
ap-northeast-1
ap-northeast-2
ap-northeast-3
ap-south-1
ap-south-2
ap-southeast-1
ap-southeast-2
ap-southeast-3
ca-central-1
eu-central-1
eu-central-2
eu-north-1
eu-south-1
eu-south-2
eu-west-1
eu-west-2
eu-west-3
me-central-1
me-south-1
sa-east-1
us-east-1
us-east-2
us-west-1
us-west-2

Follow the below steps to configure access to an Amazon S3 bucket:

  • Create a new IAM role in AWS.
  • Select Custom trust policy as the trusted entity type.
  • Copy and paste the following trust policy:
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "accounts.google.com"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "accounts.google.com:sub": "109477913720839849913"
        }
      }
    }
  ]
}
  • Copy and paste the following trust policy:
    Grant the following permissions policies to the role:
    Replace YOUR_AWS_BUCKET_NAME with the name of your AWS S3 bucket.
{
  "Version": "2012-10-17",
  "Statement": [
      {
          "Effect": "Allow",
          "Action": [
              "s3:GetObject",
              "s3:ListBucket",
              "s3:GetBucketLocation"
          ],
          "Resource": [
              "arn:aws:s3:::YOUR_AWS_BUCKET_NAME/*",
              "arn:aws:s3:::YOUR_AWS_BUCKET_NAME"
          ]
      }
  ]
}
  • Assign a name to the role and create the role.
  • Once created, view the role details to retrieve the Amazon Resource Name (ARN) and provide this to Scope3. Note this value; it has the format arn:aws:iam::**AWS_ACCOUNT**:role/**ROLE_NAME**

Items to send to Scope3:

  • S3 bucket URI, e.g. s3://example-bucket-name
  • The role ARN created above, e.g arn:aws:iam::**AWS_ACCOUNT**:role/**ROLE_NAME**
  • Optional: A sub-folder in the bucket all items will be stored in if different from the root directory
  • Optional: A date in which to only synchronize files that are created/modified after

Write permissions (optional)

As well as reports being available in the CSP, data can be returned to your cloud bucket. Please add the following bucket policy to make this possible:

{
"Version": "2012-10-17",
"Statement": [
        {
            "Sid": "Scope3 Data Sync access permissions",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::948454267882:user/scope3-data-sync-service"
            },
            "Action": [
								"s3:ListBucket",
                "s3:GetObject",
                "s3:PutObject"
            ],

            "Resource": [
                "arn:aws:s3:::YOUR_AWS_BUCKET_NAME",
                "arn:aws:s3:::YOUR_AWS_BUCKET_NAME/*"
            ]
        }
    ]
}

Make sure to swap "YOUR_AWS_BUCKET_NAME" with your bucket name.


Microsoft Azure Blob Storage

What you will need to get started:

Supported Azure Blob Storage regions:

Americas: 
East US
East US 2
West US
West US 2
West US 3
Central US
North Central US
South Central US
West Central US
Canada Central
Canada East
Brazil South

Asia-Pacific:
Australia Central
Australia East
Australia Southeast
Central India
South India
West India
Southeast Asia
East Asia
Japan East
Japan West
Korea South
Korea Central

Europe, Middle East, Africa (EMEA):
France Central
Germany West Central
Norway East
Sweden Central
Switzerland North
North Europe
West Europe
UK South
UK West
Qatar Central
UAE North
South Africa North

Follow the below steps to configure access to a Microsoft Azure Storage container:

  • Create or use an existing Microsoft Azure Storage user to access the storage account for your Microsoft Azure Storage Blob container.
  • Create a SAS (shared access signature) token at the container level. See Grant limited access to Azure Storage resources using shared access signatures for instructions.
    • The Allowed services must include Blob.
    • For Allowed resource types, select Container and Object.
    • The Allowed permissions must include Read and List
    • The default expiration time for SAS tokens is 8 hours. Set a reasonable expiration time that enables you to successfully complete your transfer. We recommend at least 6 months.
    • Do not specify any IP addresses in the Allowed IP addresses field. Storage Transfer Service uses various IP addresses and doesn't support IP address restrictions.
    • The Allowed protocols should be HTTPS only.
    • Once the token is created, note the SAS token value that is returned. You need this value when communicating with Scope3.

Items to send to Scope3:

  • Storage account name
  • Storage container name
  • The SAS (shared access signature) created above
  • Optional: A sub-folder in the bucket all items will be stored in if different from the root directory
  • Optional: A date in which to only synchronize files that are created/modified after

Write permissions (optional)

As well as reports being available in the CSP, data can be returned to your cloud bucket.

  • The Allowed permissions must include Read, List, Create, and Write

🚧

Basic SAS tokens cannot be revoked. The only way to invalidate a basic SAS token is to remove your account's storage access key. We strongly recommend that you create SAS tokens from stored access policies so that you can revoke a policy to invalidate a SAS token. See best practices when using SAS for more information.

Updated about 1 month ago