Versions Compared

Old Version 6

changes.mady.by.user Simon Guest

Saved on

compared with

New Version Current

changes.mady.by.user Thomas Berger

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

From a Windows client

S: \\storage.eri.agresearch.co.nz\datasets

M: \\storage.eri.agresearch.co.nz\projects

can be accessed as network location using the options “Add a network location”, “Quick access”, or “Map network drive…”. in the Windows File Explorer while connected to the AgResearch LAN (or via VPN).

For standard workstations, Group Policy will automatically map these with drive letters as above.

...

What is the difference between a dataset and a project provisioning type?

Excerpt
nameWhat is the difference between a dataset and a project provisioning type?

Both types are filesets on the filesystem (storage).
The dataset consists of one fileset in /tdc_persist/datasets/.
The project consists of two filesets, one in /tdc_persist/projects/ and the other in /tdc_scratch/projects/.

On AgR eRI login or compute nodes

Symbolic links are:

/agr/projects
/agr/datasets
/agr/scratch

Links are:
/tdc_persist/projects
/tdc_persist/datasets
/tdc_scratch/projects
Mount points are:

/mnt/gpfs/persist/ for both datasets and projects and
/mnt/gpfs/scratch/ for projects temp. working space

See also How to access the AgResearch eRI compute cluster

Locating legacy data on eRI

In order to ease the transition from HPC to eRI, a legacy link farm has been created, with symbolic links from the legacy paths to the new locations. This is mounted on all eRI nodes (login and compute).

This addresses the problem of the very many legacy scripts which have hardcoded paths to the legacy data locations.For example (fake for now, anticipating these links appearing imminently!)

Info

active -> /agr/persist/projects/{uniqueId}/active
scratch -> /agr/persist/projects/{uniqueId}/scratch
itmp -> /agr/scratch/projects/{uniqueId}
archive -> /agr/persist/datasets/{uniqueId}

Example:

Code Block
login-0$ ls -l /dataset/blastdata/
total 2
lrwxrwxrwx. 1 eri_migration@iam.flexi.nesi.org.nz eri_migration@iam.flexi.nesi.org.nz 43 Jul 18 16:29 active -> /agr/persist/projects/2002-blastdata/active
lrwxrwxrwx. 1 eri_migration@iam.flexi.nesi.org.nz eri_migration@iam.flexi.nesi.org.nz 3644 Jul 18 16:29 archivescratch -> /agr/persist/datasetsprojects/2002-blastdata/scratch
lrwxrwxrwx. 1 eri_migration@iam.flexi.nesi.org.nz eri_migration@iam.flexi.nesi.org.nz 36 Jul 18 16:29 itmp -> /agr/scratch/projects/2002-blastdata
lrwxrwxrwx. 1 eri_migration@iam.flexi.nesi.org.nz eri_migration@iam.flexi.nesi.org.nz 4436 Jul 18 16:29 scratcharchive -> /agr/persist/projectsdatasets/2002-blastdata/scratch

login-0$ ls -l /dataset/blastdata/active/
454temp                    hs_faa_bak                                  plant.protein.faa.psi            swissprot.00.pnd  temp                            UniVec.nhr
agall.seq.exists           junkx1                                      plant.protein.faa.psq            swissprot.00.pni  tigr                            UniVec.nin
agfilter1.pl               ln_mirror.sh                                plant.rna.fna.nhr                swissprot.00.pog  Tobacco_MF_assembl.exists       UniVec.nsd
bgi_sheep                  ln_mirror.sh.bu1                            plant.rna.fna.nin                swissprot.00.ppd  Tobacco_MF_assembly.fa.exists   UniVec.nsi
blastdata.exists           log                                         plant.rna.fna.nnd                swissprot.00.ppi  UMD3_OA_v1                      UniVec.nsq
BTA_OA_ver.2.exists        merged.dmp                                  plant.rna.fna.nni                swissprot.00.psd  uniprot_kb.fa                   UniVec.prop
bt_faa_bak                 mirror                                      plant.rna.fna.nsd                swissprot.00.psi  uniprot_kb.fasta.exists         unpack.sh
citations.dmp              names.dmp                                   plant.rna.fna.nsi                swissprot.00.psq  uniprot_sprot.fa.exists         vector.fa
cs08.seq.exists            nodes.dmp                                   plant.rna.fna.nsq                swissprot.pal     uniprot_swissprot.fasta.exists  vector.nhr
delnodes.dmp               OA_chromosomes_ver.1.0                      public_readonly                  taxbti.bti.bak    UniVec                          vector.nin
division.dmp               OAR_chromosomes_ver.1.0.exists              readme.txt                       taxdb.btd         UniVec_Core.exists              vector.nnd
est.exists                 OARv3.0_masked_with_SNPs_and_indels.exists  reorg1.sh.bu1                    taxdb.btd.bu1     UniVec_Core.fa.exists           vector.nni
gc.prt                     obsolete_deleteafter01112008                riceensembl                      taxdb.bti         UniVec_Core.nhr                 vector.nsd
gencode.dmp                plant.protein.faa.phr                       sheep_chr_OAR.exists             taxdb.bti.bak     UniVec_Core.nin                 vector.nsi
geneious_blast_template    plant.protein.faa.pin                       sheep.v3.0.14th.final.fa.exists  taxdb.bti.bu1     UniVec_Core.nsd                 vector.nsq
geneious_blast_template.0  plant.protein.faa.pnd                       species                          taxdb.tar.gz      UniVec_Core.nsi                 vector.tar.gz
geneious_blast_testdb      plant.protein.faa.pni                       stampfiles                       taxdb.tar.gz.1    UniVec_Core.nsq                 Wrightson_ESTs.exists
gi                         plant.protein.faa.pog                       swissprot.00.phr                 taxdb.tar.gz.2    UniVec_Core.prop
hg17                       plant.protein.faa.psd                       swissprot.00.pin                 taxdump.tar.gz    UniVec.exists

...

From a Windows client
S: \\storage.eri.agresearch.co.nz\datasets s drive
M: \\storage.eri.agresearch.co.nz\projects m drive
For standard workstations, a Group Policy will automatically map these with drive letters as above.
It can also be accessed as network location using the options “Add a network location”, “Quick access”, or “Map network drive…”. in the Windows File Explorer while connected to the AgResearch LAN (or via VPN).
 Filter by label (Content by label)
showLabelsfalse
showSpacefalse
cqllabel in ( "kb-info-article" , "kb-how-to-article" ) and label = "access" and type = "page" and space = currentSpace ( )
...