Configuring GCP Accounts at the Organization Level using gcloud

To configure the GCP billing account using gcloud CLI, you need the project ID of a project assigned to that billing account and your organization ID. When you complete configuration, all projects assigned to the billing account are pulled into the CloudHealth Platform.

If you have already enabled BigQuery, use the project that contains your BigQuery dataset. Otherwise, you can use any project assigned to the billing account.

To ensure that billing data isn't lost, use the project ID of a project that won't be deleted.

The project ID is located in the Project Info pane of the Dashboard in the Google Cloud Console.

The organization ID is located in IAM & admin > Manage resources in the Google Console.

Enter the following command to create a custom role with the permissions CloudHealth requires at the organization level.

• Replace <insert org id> with the ID of your organization.
• Replace <insert custom role name> with the name of the new custom role.

    gcloud iam roles create <insert custom role name> --organization <insert org id> \
    --title <insert custom role name> --description \
    "Used to collect CloudHealth Billing and Usage data" --permissions \
    resourcemanager.projects.get,compute.disks.get,compute.disks.list,compute.images.get,compute.images.list,compute.instances.get,compute.instances.list,compute.projects.get,compute.regions.get,compute.regions.list,storage.buckets.get,storage.buckets.list,bigquery.datasets.get,bigquery.jobs.create,dataproc.clusters.list,compute.addresses.list,container.clusters.list,bigquery.tables.get,bigquery.tables.getData,compute.snapshots.get,compute.snapshots.list,compute.zones.get,compute.zones.list,recommender.computeInstanceMachineTypeRecommendations.get,recommender.computeInstanceMachineTypeRecommendations.list,recommender.computeInstanceMachineTypeRecommendations.update,recommender.locations.get,recommender.locations.list \
    --stage GA

If you have already enabled BigQuery in the Google Console, skip this step.

Enter the following command in the gcloud CLI to create a dataset to use later when enabling BigQuery.

• Replace <insert project ID> with your project ID.
• Replace <insert dataset name> with your dataset name.

bq mk <insert project id>:<insert dataset name>

Enter the following command in the gcloud CLI to allow CloudHealth to gather cost and tagging information for your projects. This command enables the following APIs:

• Google Compute Engine API
• Google Cloud Storage API
• Google Cloud Resource Manager API
• Google Cloud Storage JSON API
• Recommender API
• Kubernetes Engine API
• Dataproc API
• BigQuery API (See Step 5)

for project in $(gcloud projects list --format="value(projectId)")
do
if [[ $(gcloud beta billing projects describe $project --format="value(billingEnabled)") = "True" ]];
then
echo "ProjectId: $project - Enabling APIs..."
gcloud services enable compute.googleapis.com \
cloudresourcemanager.googleapis.com \
storage-component.googleapis.com \
recommender.googleapis.com \
container.googleapis.com \
dataproc.googleapis.com \
--project $project
fi;
done

This gcloud command only enables the APIs for existing projects. If you create new projects in the future, you must manually enable the above APIs for the new projects. Allow about 32 hours for the platform to collect and process the data from GCP to project the cost.

Enter the following command in the gcloud CLI to allow CloudHealth to gather data from your BigQuery dataset. Replace <insert project ID> with your project ID.

gcloud services enable bigquery-json.googleapis.com --project <insert project id>

Enter the following command in the gcloud CLI to create a service account.

• Replace <insert project ID> with your project ID.
• Replace <insert service account name> with your new service account name.

gcloud iam service-accounts create <insert service account name> --project <insert project id>

Enter the following command in the gcloud CLI to create a private key for the newly created service account and to save the key to the $HOME directory of your instance.

• Replace <insert project ID> with your project ID.
• Replace <insert service account name> with your service account name.
• Replace <insert private key name> with your new private key name.

gcloud iam service-accounts keys create \
--iam-account <insert service account name>@<insert project id>.iam.gserviceaccount.com <insert private key name>.json

Enter the following command in the gcloud CLI to download the service account key to your local machine. Replace <insert private key name> with your private key name.

cloudshell download <insert private key name>.json

Enter the following command to assign the custom role you created in Step 2 to the service account.

• Replace <insert service account name> with your service account name.
• Replace <insert project ID> with your project ID.
• Replace <insert org id> with your organization ID.
• Replace <insert custom role name> with the custom role name.

gcloud projects add-iam-policy-binding <insert project id> --member serviceAccount:<insert service account name>@<insert project id>.iam.gserviceaccount.com \
--role organizations/<insert org id>/roles/<insert custom role name>

Create an IAM member for your organization with the custom role you created in Step 2.

• Replace <insert service account name> with your service account name.
• Replace <insert org id> with the ID of your organization.
• Replace <insert project ID> with your project ID.
• Replace <insert custom role name> with the name of the new custom role.

gcloud organizations add-iam-policy-binding <insert org id> --member user:<insert service account name>@<insert project id>.iam.gserviceaccount.com \
    --member serviceAccount:<insert service account name>@<insert project id>.iam.gserviceaccount.com \
--role organizations/<org id>/roles/<insert custom role name>

Enabling BigQuery can only be completed in the Google Console.

Use the dataset you created previously as your BigQuery dataset. If you have already enabled BigQuery in the Google Console, skip this step.

You need to configure only the billing account. CloudHealth discovers all derived projects associated with the billing account that have incurred costs.

1. Log in to the CloudHealth platform and from the left menu, select Setup > Accounts > GCP Billing. Then click Add Account.

2. Enter your billing account, BigQuery, billing export, and service account information into the form. If needed, you can locate this information in the Google Cloud Console.

3. In Service Account section, select Choose file and upload the JSON private key.

   When you use a JSON key, CloudHealth verifies that the project ID in the JSON key matches the ID of the project to which you are attaching the credentials.

4. If you want to use different service accounts to collect billing statements, select the Enable a separate service account for projects linked to this billing account. (Optional) checkbox. Select Choose file and upload the JSON private key.

   This option should be used if you want to use different service accounts for billing data collection and asset metadata collection.

5. Click Save Account.

CloudHealth validates new Google Service Accounts and derived projects every 4 hours. You can view derived projects by going to Setup > Accounts > GCP Project. Projects you enable in the CloudHealth Platform change status from Not Configured to Green, Yellow, Red, or Pending.

The newly configured GCP billing accounts will be onboarded automatically, and within 48 hrs, you will see the cost data in the CloudHealth platform.