Can I share bucket files between different workspaces?

  • Updated

You are capable of copying files from one workspace notebook to another workspace’s Workspace Bucket. You are also able to copy files from one Workspace Bucket to another Workspace Bucket in a different workspace using gsutil. In the link below you have additional information about the basics to gsutil.

Copy, rename, and move objects  |  Cloud Storage  |  Google Cloud

NOTE: the workspaces must share the same data tier access level. For example, you can only access bucket files derived from Controlled Tier workspaces using another Controlled Tier workspace. 

Below are some sample scenarios: 

From workspace_bucket1 in workspace1 to workspace_bucket2 in workspace2:

The code below can be used in a workspace notebook of destination or origin. In this code we are using workspace buckets and we can copy several documents or files under one folder with the code below (please note you need to have permission to access applicable buckets): 

You will need a path of origin and destination of the files. You need the name of your workspace buckets in both workspaces:

BUCKET1=os.getenv('WORKSPACE_BUCKET') # in workspace 1
BUCKET2=os.getenv('WORKSPACE_BUCKET') # in workspace 2
  • Copying different files in one folder: 
 !gsutil -m cp -r {BUCKET1}/notebooks/data  {BUCKET2}/notebooks

In this case, any file under the folder data from the BUCKET1 will be copied in the BUCKET2 under notebooks. With gsutil you can create the folders in your destination bucket even if they do not exist initially.

  • Copying one specific file: 

If you just need to copy one file for example called hello.txt under data file in BUCKET1 to BUCKET2:

!gsutil  cp  {BUCKET1}/notebooks/data/hello.txt   {BUCKET2}/notebooks/hello.txt

From workpsace1 in VM to workspace bucket in workpsace2:

  • Copying file 1 from workspace using  a notebook in workpsace1: 

If the file hello2.txt is in your VM, you need  to find the file location or path to the file or folder you need to copy; the current working directory  should be your workspace: /home/Jupiter/workspaces/workspace1. You can find the files under your directory with the following code:

%%bash
ls  /home/jupyter/workspaces/workspace1/*

you can define a variable environment with the following code to your path:

%env path = /home/jupyter/workspaces/ workspace1/hello2.txt

if your bucket2 is called:

BUCKET2=’gs://fc-secure-84f4c31e-92dd-4e0e-ba3e-2edbfcaf377e’

If we are copying the file in the BUCKET2 in a different workspace the code is the following:

!gsutil  cp  $path  'gs://fc-secure-84f4c31e-92dd-4e0e-ba3e-2edbfcaf377e'/notebooks/ hello2.txt
  • Copying files under a folder from a workspace using a notebook in workspace1: 

If you want to copy a folder with different files:

%env path2 = /home/jupyter/workspaces/workspace1/folder
!gsutil -m cp -r $path2  'gs://fc-secure-84f4c31e-92dd-4e0e-ba3e-2edbfcaf377e'/notebooks

You could see that the files are copied on your workspace2 by using the code on a notebook in workspace2:

!gsutil ls {BUCKET2}/notebooks/*

 

Was this article helpful?

0 out of 1 found this helpful

Have more questions? Submit a request

Comments

0 comments

Article is closed for comments.