amdapy package¶

Submodules¶

amdapy.amda module¶

author: Alexandre Schulz
email: alexandre.schulz@irap.omp.eu
brief: AMDA dataset provider

amdapy.amda.AMDA is the object through witch we can query the AMDA database for available missions, instruments and datasets. Datasets are stored following a predefined hierarchy:

AMDA collection 
    Mission A
        Instrument X
            Dataset 1
            ...
            Dataset N
        ...
    ...

class amdapy.amda.AMDA[source]¶

Bases: object

AMDA database connector. Use this object to connect to query AMDAs database. The collection (Collection) allows acces to dataset description.

>>> amda = amdapy.amda.AMDA()
>>> for dataset in amda.collection.iter_dataset():
>>>     print(dataset)

get(item, start=None, stop=None, sampling=None)[source]¶

Gets a item from the collection.

Parameters

item (amdapy.amda._CollectionItem) – collection item
start (datetime.datetime or str) – data start time
stop (datetime.datetime or str) – data stop time
sampling (float) – sampling in seconds

Returns

parameter or dataset depending on the input, None if item is badly defined

Return type

amda.Parameter or amda.Dataset

The behaviour of this method depends on the type of the item argument

amdapy.amda.Collection.Dataset then call the amdapy.amda.AMDA.get_dataset()
amdapy.amda.Collection.Parameter then call the amdapy.amda.AMDA.get_parameter()

For example retriving the tao-ura-sw dataset is done like this :

>>> dataset_desc = amda.find("tao-ura-sw")
>>> data = amda.get(dataset_desc)
>>> data
Dataset (id:tao-ura-sw, start:2010-01-01 00:00:00, stop:2021-02-19 00:00:00, n_param:7)
    Parameter (id:ura_sw_n, name:density, units:cm⁻³, value: None)
    Parameter (id:ura_sw_v, name:velocity, units:km/s, value: None)
    Parameter (id:ura_sw_t, name:temperature, units:eV, value: None)
    Parameter (id:ura_sw_pdyn, name:dynamic pressure, units:nPa, value: None)
    Parameter (id:ura_sw_b, name:b tangential, units:nT, value: None)
    Parameter (id:ura_sw_bx, name:b radial, units:nT, value: None)
    Parameter (id:ura_sw_da, name:angle Uranus-Sun-Earth, units:deg, value: None)

The start and stop attributes indicate the desired begining and end of the data. If they are str objects then they must follow the following scheme

YYYY-MM-DDThh:mm:ss

get_dataset(dataset_item, start=None, stop=None, sampling=None)[source]¶

Get dataset contents.

Parameters

dataset_item – dataset item
start (datetime.datetime) – data start time
stop (datetime.datetime) – data stop time
sampling (float) – sampling in seconds

Returns

dataset object if found, None otherwise

Return type

amdapy.amda.Dataset or None

Retrieves the dataset contents between t_interval.start to t_interval.stop. If the t_interval is not a valid interval then only the first day of data will be retrieved. This behaviour was chosen to avoid too many requests for downloading the whole dataset (which can be extensive).

In the following example we download the first day of the tao-ura-sw.

>>> amda = AMDA()
>>> dataset_desc = amda.collection.find("tao-ura-sw")
>>> dataset = amda.get_dataset(dataset_desc)
>>> dataset
Dataset (id:tao-ura-sw, start:2010-01-01 00:00:00, stop:2021-02-19 00:00:00, n_param:7)
    Parameter (id:ura_sw_n, name:density, units:cm⁻³, shape: (24,))
    Parameter (id:ura_sw_v, name:velocity, units:km/s, nodata)
    Parameter (id:ura_sw_t, name:temperature, units:eV, shape: (24,))
    Parameter (id:ura_sw_pdyn, name:dynamic pressure, units:nPa, shape: (24,))
    Parameter (id:ura_sw_b, name:b tangential, units:nT, shape: (24,))
    Parameter (id:ura_sw_bx, name:b radial, units:nT, shape: (24,))
    Parameter (id:ura_sw_da, name:angle Uranus-Sun-Earth, units:deg, shape: (24,))

The actual data is stored as a pandas.DataFrame object. You can access the data through the bracket operator. When passing a slice object to amdapy.amda.AMDA.get_dataset() the bracket operator is called on the data attribute.

>>> dataset[:]
                           density  velocity_V r  ...  b radial  angle Uranus-Sun-Earth
Time                                            ...                                  
2010-01-01T01:00:00.000    0.005       347.811  ...    -0.002                   2.305
2010-01-01T02:00:00.000    0.006       347.616  ...    -0.002                   2.518
2010-01-01T03:00:00.000    0.006       347.407  ...    -0.002                   2.731
2010-01-01T04:00:00.000    0.007       347.185  ...    -0.002                   2.938
2010-01-01T05:00:00.000    0.008       346.966  ...    -0.002                   3.148
2010-01-01T06:00:00.000    0.008       346.709  ...    -0.002                   3.360
2010-01-01T07:00:00.000    0.008       346.539  ...    -0.002                   3.571

The previous call is equivalent to :

>>> dataset.data[:]

Passing a parameter name or id to amdapy.amda.AMDA.get_dataset() will return the corresponding amdapy.amda.Parameter object. Individual parameters are retrieved by :

>>> dataset["density"]
Parameter (id:ura_sw_n, name:density, units:cm⁻³, shape: (24,))

get_derived(userid, password, paramid, start, stop, sampling=None, col_names=None)[source]¶

get_parameter(param, start=None, stop=None, sampling=None)[source]¶

Get parameter data.

Parameters

param (amda.Collection.Parameter) – parameter descriptor
start (datetime.datetime) – data start time
stop (datetime.datetime) – data stop time
sampling (float) – sampling time in seconds

Returns

Parameter object

Return type

amdapy.amda.Parameter

Given a valid amdapy.amda.Collection.Parameter instance you can retrieve the parameters data by calling the amdapy.amda.AMDA.get_parameter() method on the desired time interval.

If the time interval provided is not valid then only the first day of data will be returned. Data is stored as a pandas.DataFrame object indexed by time (since all AMDA parameters are timeseries), and you can access it through the data attribute.

Continuing with the example dataset tao-ura-sw you can retrive the density parameter (id : ura_sw_n) like this :

>>> parameter_desc = amda.collection.find("ura_sw_n")
>>> parameter_desc
Parameter item (id:ura_sw_n, name:density, units:cm⁻³, disp:None, dataset:tao-ura-sw, n:1)
>>> parameter = amda.get_parameter(parameter_desc)
>>> parameter
Parameter (id:ura_sw_n, name:density, units:cm⁻³, shape: (24, 1))
>>> parameter_desc[:]
                     density
Time                        
2010-01-01 01:00:00    0.005
2010-01-01 02:00:00    0.006
...
2010-01-01 23:00:00    0.008
2010-01-02 00:00:00    0.008

list_derived(userid, password)[source]¶

class amdapy.amda.Collection[source]¶

Bases: object

The Collection object is used for getting descriptions of the items in AMDAs database. Navigation the database is done with the Collection.iter_dataset() iterator.

class Dataset(id, name, parameters=[], globalstart=None, globalstop=None)[source]¶

Bases: amdapy.amda._CollectionItem

This object contains a description of a dataset. Attributes are:

id : unique identifier for the dataset
name : name of the dataset in AMDAs navigation tree
globalstart: data start time (datetime.datetime )
globalstop: data stop time (datetime.datetime)
parameters : list of parameter descriptions (amdapy.amda.Collection.Parameter)
n : number of parameters

Parameters

id (str) – dataset identification, should be unique, used for retriving contents of the dataset
name (str) – name of the dataset
parameters (list of amdapy.amda.Collection.Parameter objects) – list of parameter objects belonging to the dataset
globalstart (datetime.datetime) – start time
globalstop (datetime.datetime) – stop time

This object contains the descriptions of a dataset available in AMDA. You can acces the datasets unique identifier through the id attribute that is common to all items of the collection. Datasets are then defined by a name, a description string, a list of parameters (parameter description objects of type amdapy.amda.Collection.Parameter.

Here is an example of a dataset description string

Dataset (id:tao-ura-sw, start:2010-01-01 00:00:00, stop:2021-02-19 00:00:00, n_param:7)
   Parameter (id:ura_sw_n, name:density, units:cm⁻³, shape: (24,))
   Parameter (id:ura_sw_v, name:velocity, units:km/s, nodata)
   Parameter (id:ura_sw_t, name:temperature, units:eV, shape: (24,))
   Parameter (id:ura_sw_pdyn, name:dynamic pressure, units:nPa, shape: (24,))
   Parameter (id:ura_sw_b, name:b tangential, units:nT, shape: (24,))
   Parameter (id:ura_sw_bx, name:b radial, units:nT, shape: (24,))
   Parameter (id:ura_sw_da, name:angle Uranus-Sun-Earth, units:deg, shape: (24,))

This object is intended to provide a description of the dataset containing parameter identifiers, name, units and other important information. Downloading the dataset is done by passing a amdapy.amda.Collection.Dataset object to the amdapy.amda.AMDA.get_dataset() methode.

class Parameter(id, name, units, description, displaytype, dataset_id, components=[])[source]¶

Bases: amdapy.amda._CollectionItem

This object contains parameter descriptions.

Warning

Parameter descriptions do not contain any information about the data timespan, to get the begining and end date of the data you must retrieve the parent dataset whose id is given by the dataset_id attribute.

You can access the following information through this container

id : unique parameter id
name : parameter name
units : data units
dataset_id : parent dataset id
components : list of components, only if these are available in AMDA

Parameters

id (str) – parameter identification
name (str) – name of the parameter
units (str) – units
description (str) – parameter description
displaytype (str) – parameter display type
dataset_id (str) – identification of the parent dataset
components (list of amdapy.amda.Collection._Component objects) – list of components

find(id)[source]¶

Find and collection item by id.

Parameters: id (str) – id of the desired item
Returns: Collection item with the right id if found, None otherwise
Return type: amdapy.amda._CollectionItem or None

Iterates over all dataset objects in search for one with the right id, if the id doesn’t match then proceeds to check all parameters in the dataset before moving on to the next.

>>> amda = amdapy.amda.AMDA()
>>> var = amda.find("ura_sw_da")
>>> var
Parameter item (id:ura_sw_da, name:angle Uranus-Sun-Earth, units:deg, disp:None, dataset:tao-ura-sw, n:1)

If we want a description of the dataset this parameter belongs to then we can do :

>>> amda.collection.find(var.dataset_id)
Dataset item (id:tao-ura-sw, name:SW / Input OMNI, start:2010-01-01 00:00:00, stop:2021-02-19 00:00:00, n_param:7)h
    Parameter item (id:ura_sw_n, name:density, units:cm⁻³, disp:None, dataset:tao-ura-sw, n:1)
    Parameter item (id:ura_sw_v, name:velocity, units:km/s, disp:None, dataset:tao-ura-sw, n:2)
    Parameter item (id:ura_sw_t, name:temperature, units:eV, disp:None, dataset:tao-ura-sw, n:1)
    Parameter item (id:ura_sw_pdyn, name:dynamic pressure, units:nPa, disp:None, dataset:tao-ura-sw, n:1)
    Parameter item (id:ura_sw_b, name:b tangential, units:nT, disp:None, dataset:tao-ura-sw, n:1)
    Parameter item (id:ura_sw_bx, name:b radial, units:nT, disp:None, dataset:tao-ura-sw, n:1)
    Parameter item (id:ura_sw_da, name:angle Uranus-Sun-Earth, units:deg, disp:None, dataset:tao-ura-sw, n:1)

The object returned by amdapy.amda.Collection.find() can be passed to the amdapy.amda.AMDA.get() method to retrive the contents of the parameter or dataset.

iter_dataset()[source]¶

Collection item iterator

Returns: collection items
Return type: amdapy.amda.Collection.Item

class amdapy.amda.Dataset(el, data)[source]¶

Bases: object

AMDA dataset container. Retrieving data from AMDA can be time consuming, as such the object constructor can be called without setting the data.

Parameters

el (amdapy.amdaWSClient.client.DatasetElement) – dataset unique identificator
data (array type) – optional (default: None), dataset content

Get list of parameter ids :

>>> [p.id for p in dataset.iter_parameter()]
['ura_sw_n', 'ura_sw_v', 'ura_sw_t', 'ura_sw_pdyn', 'ura_sw_b', 'ura_sw_bx', 'ura_sw_da']

You can then get the contents of the parameters by using the bracket operator :

>>> dataset["ura_sw_n"]

iter_parameter()[source]¶

Parameter iterator

Returns: parameter object
Return type: amdapy.amda.Parameter

Parameter iterator. For example get a list of parameter name and units :

>>> [p.name for p in dataset.iter_parameter()]
['density', 'velocity', 'temperature', 'dynamic pressure', 'b tangential', 'b radial', 'angle Uranus-Sun-Earth']
>>> [p.units for p in dataset.iter_parameter()]
['cm⁻³', 'km/s', 'eV', 'nPa', 'nT', 'nT', 'deg']

make_parameter_list(el)[source]¶

time()[source]¶

Get time value for this dataset

Returns: time
Return type: pandas.DataFrame

Get the datasets time values.

>>> T = dataset.time()
>>> type(T)
<class 'pandas.core.indexes.datetimes.DatetimeIndex'>
>>> T
DatetimeIndex(['2010-01-01 01:00:00', '2010-01-01 02:00:00',
               '2010-01-01 03:00:00', '2010-01-01 04:00:00',
...
               '2010-01-01 23:00:00', '2010-01-02 00:00:00'],
              dtype='datetime64[ns]', name='Time', freq=None)

class amdapy.amda.Parameter(id, name, units, data)[source]¶

Bases: object

Container class for storing parameter objects.

Parameters

id (str) – identifier of the parameter, should be unique
name (str) – name of the parameter as seen in the AMDA navigation tree
units (str) – units in which are expressed the parameters values
data (pandas.DataFrame) – values of the parameter

This class is a container for parameter data including identification, name, units and data. We retrieve amdapy.amda.Parameter instances by passing a amdapy.amda.Collection.Parameter object to the amdapy.amda.AMDA.get() or amdapy.amda.AMDA.get_dataset() method.

>>> parameter
Parameter (id:ura_sw_n, name:density, units:cm⁻³, shape: (24, 1))

plot(dataset_id=None, figsize=None)[source]¶

Plot the parameter. Naive plotting function for visualizing time series data.

Returns: pyplot figure object
Return type: pyplot figure

amdapy package¶

Submodules¶

amdapy.amda module¶

Module contents¶

Table of Contents

Previous topic

Next topic

This Page