amdapy package

Submodules

amdapy.amda module

author

Alexandre Schulz

email

alexandre.schulz@irap.omp.eu

brief

AMDA dataset provider

amdapy.amda.AMDA is the object through witch we can query the AMDA database for available missions, instruments and datasets. Datasets are stored following a predefined hierarchy:

AMDA collection 
    Mission A
        Instrument X
            Dataset 1
            ...
            Dataset N
        ...
    ...
class amdapy.amda.AMDA[source]

Bases: object

AMDA database connector. Use this object to connect to query AMDAs database. The collection (Collection) allows acces to dataset description.

>>> amda = amdapy.amda.AMDA()
>>> for dataset in amda.collection.iter_dataset():
>>>     print(dataset)
get(item, start=None, stop=None)[source]

Gets a item from the collection.

Parameters
  • item (amdapy.amda._CollectionItem) – collection item

  • start (datetime.datetime or str) – data start time

  • stop (datetime.datetime or str) – data stop time

Returns

parameter or dataset depending on the input, None if item is badly defined

Return type

amda.Parameter or amda.Dataset

The behaviour of this method depends on the type of the item argument

For example retriving the tao-ura-sw dataset is done like this :

>>> dataset_desc = amda.find("tao-ura-sw")
>>> data = amda.get(dataset_desc)
>>> data
Dataset (id:tao-ura-sw, start:2010-01-01 00:00:00, stop:2021-02-19 00:00:00, n_param:7)
    Parameter (id:ura_sw_n, name:density, units:cm⁻³, value: None)
    Parameter (id:ura_sw_v, name:velocity, units:km/s, value: None)
    Parameter (id:ura_sw_t, name:temperature, units:eV, value: None)
    Parameter (id:ura_sw_pdyn, name:dynamic pressure, units:nPa, value: None)
    Parameter (id:ura_sw_b, name:b tangential, units:nT, value: None)
    Parameter (id:ura_sw_bx, name:b radial, units:nT, value: None)
    Parameter (id:ura_sw_da, name:angle Uranus-Sun-Earth, units:deg, value: None)

The start and stop attributes indicate the desired begining and end of the data. If they are str objects then they must follow the following scheme

YYYY-MM-DDThh:mm:ss
get_dataset(dataset_item, start=None, stop=None)[source]

Get dataset contents.

Parameters
  • dataset_item – dataset item

  • start (datetime.datetime) – data start time

  • stop (datetime.datetime) – data stop time

Returns

dataset object if found, None otherwise

Return type

amdapy.amda.Dataset or None

Retrieves the dataset contents between t_interval.start to t_interval.stop. If the t_interval is not a valid interval then only the first day of data will be retrieved. This behaviour was chosen to avoid too many requests for downloading the whole dataset (which can be extensive).

In the following example we download the first day of the tao-ura-sw.

>>> amda = AMDA()
>>> dataset_desc = amda.collection.find("tao-ura-sw")
>>> dataset = amda.get_dataset(dataset_desc)
>>> dataset
Dataset (id:tao-ura-sw, start:2010-01-01 00:00:00, stop:2021-02-19 00:00:00, n_param:7)
    Parameter (id:ura_sw_n, name:density, units:cm⁻³, shape: (24,))
    Parameter (id:ura_sw_v, name:velocity, units:km/s, nodata)
    Parameter (id:ura_sw_t, name:temperature, units:eV, shape: (24,))
    Parameter (id:ura_sw_pdyn, name:dynamic pressure, units:nPa, shape: (24,))
    Parameter (id:ura_sw_b, name:b tangential, units:nT, shape: (24,))
    Parameter (id:ura_sw_bx, name:b radial, units:nT, shape: (24,))
    Parameter (id:ura_sw_da, name:angle Uranus-Sun-Earth, units:deg, shape: (24,))

The actual data is stored as a pandas.DataFrame object. You can access the data through the bracket operator. When passing a slice object to amdapy.amda.AMDA.get_dataset() the bracket operator is called on the data attribute.

>>> dataset[:]
                           density  velocity_V r  ...  b radial  angle Uranus-Sun-Earth
Time                                            ...                                  
2010-01-01T01:00:00.000    0.005       347.811  ...    -0.002                   2.305
2010-01-01T02:00:00.000    0.006       347.616  ...    -0.002                   2.518
2010-01-01T03:00:00.000    0.006       347.407  ...    -0.002                   2.731
2010-01-01T04:00:00.000    0.007       347.185  ...    -0.002                   2.938
2010-01-01T05:00:00.000    0.008       346.966  ...    -0.002                   3.148
2010-01-01T06:00:00.000    0.008       346.709  ...    -0.002                   3.360
2010-01-01T07:00:00.000    0.008       346.539  ...    -0.002                   3.571

The previous call is equivalent to :

>>> dataset.data[:]

Passing a parameter name or id to amdapy.amda.AMDA.get_dataset() will return the corresponding amdapy.amda.Parameter object. Individual parameters are retrieved by :

>>> dataset["density"]
Parameter (id:ura_sw_n, name:density, units:cm⁻³, shape: (24,))
get_parameter(param, start=None, stop=None)[source]

Get parameter data.

Parameters
  • param (amda.Collection.Parameter) – parameter descriptor

  • start (datetime.datetime) – data start time

  • stop (datetime.datetime) – data stop time

Returns

Parameter object

Return type

amdapy.amda.Parameter

Given a valid amdapy.amda.Collection.Parameter instance you can retrieve the parameters data by calling the amdapy.amda.AMDA.get_parameter() method on the desired time interval.

If the time interval provided is not valid then only the first day of data will be returned. Data is stored as a pandas.DataFrame object indexed by time (since all AMDA parameters are timeseries), and you can access it through the data attribute.

Continuing with the example dataset tao-ura-sw you can retrive the density parameter (id : ura_sw_n) like this :

>>> parameter_desc = amda.collection.find("ura_sw_n")
>>> parameter_desc
Parameter item (id:ura_sw_n, name:density, units:cm⁻³, disp:None, dataset:tao-ura-sw, n:1)
>>> parameter = amda.get_parameter(parameter_desc)
>>> parameter
Parameter (id:ura_sw_n, name:density, units:cm⁻³, shape: (24, 1))
>>> parameter_desc[:]
                     density
Time                        
2010-01-01 01:00:00    0.005
2010-01-01 02:00:00    0.006
...
2010-01-01 23:00:00    0.008
2010-01-02 00:00:00    0.008
class amdapy.amda.Collection[source]

Bases: object

The Collection object is used for getting descriptions of the items in AMDAs database. Navigation the database is done with the Collection.iter_dataset() iterator.

class Dataset(id, name, parameters=[], globalstart=None, globalstop=None)[source]

Bases: amdapy.amda._CollectionItem

This object contains a description of a dataset. Attributes are:
  • id : unique identifier for the dataset

  • name : name of the dataset in AMDAs navigation tree

  • globalstart: data start time (datetime.datetime )

  • globalstop: data stop time (datetime.datetime)

  • parameters : list of parameter descriptions (amdapy.amda.Collection.Parameter)

  • n : number of parameters

Parameters
  • id (str) – dataset identification, should be unique, used for retriving contents of the dataset

  • name (str) – name of the dataset

  • parameters (list of amdapy.amda.Collection.Parameter objects) – list of parameter objects belonging to the dataset

  • globalstart (datetime.datetime) – start time

  • globalstop (datetime.datetime) – stop time

This object contains the descriptions of a dataset available in AMDA. You can acces the datasets unique identifier through the id attribute that is common to all items of the collection. Datasets are then defined by a name, a description string, a list of parameters (parameter description objects of type amdapy.amda.Collection.Parameter.

Here is an example of a dataset description string

Dataset (id:tao-ura-sw, start:2010-01-01 00:00:00, stop:2021-02-19 00:00:00, n_param:7)
   Parameter (id:ura_sw_n, name:density, units:cm⁻³, shape: (24,))
   Parameter (id:ura_sw_v, name:velocity, units:km/s, nodata)
   Parameter (id:ura_sw_t, name:temperature, units:eV, shape: (24,))
   Parameter (id:ura_sw_pdyn, name:dynamic pressure, units:nPa, shape: (24,))
   Parameter (id:ura_sw_b, name:b tangential, units:nT, shape: (24,))
   Parameter (id:ura_sw_bx, name:b radial, units:nT, shape: (24,))
   Parameter (id:ura_sw_da, name:angle Uranus-Sun-Earth, units:deg, shape: (24,))

This object is intended to provide a description of the dataset containing parameter identifiers, name, units and other important information. Downloading the dataset is done by passing a amdapy.amda.Collection.Dataset object to the amdapy.amda.AMDA.get_dataset() methode.

class Parameter(id, name, units, description, displaytype, dataset_id, components=[])[source]

Bases: amdapy.amda._CollectionItem

This object contains parameter descriptions.

Warning

Parameter descriptions do not contain any information about the data timespan, to get the begining and end date of the data you must retrieve the parent dataset whose id is given by the dataset_id attribute.

You can access the following information through this container
  • id : unique parameter id

  • name : parameter name

  • units : data units

  • dataset_id : parent dataset id

  • components : list of components, only if these are available in AMDA

Parameters
  • id (str) – parameter identification

  • name (str) – name of the parameter

  • units (str) – units

  • description (str) – parameter description

  • displaytype (str) – parameter display type

  • dataset_id (str) – identification of the parent dataset

  • components (list of amdapy.amda.Collection._Component objects) – list of components

find(id)[source]

Find and collection item by id.

Parameters

id (str) – id of the desired item

Returns

Collection item with the right id if found, None otherwise

Return type

amdapy.amda._CollectionItem or None

Iterates over all dataset objects in search for one with the right id, if the id doesn’t match then proceeds to check all parameters in the dataset before moving on to the next.

>>> amda = amdapy.amda.AMDA()
>>> var = amda.find("ura_sw_da")
>>> var
Parameter item (id:ura_sw_da, name:angle Uranus-Sun-Earth, units:deg, disp:None, dataset:tao-ura-sw, n:1)

If we want a description of the dataset this parameter belongs to then we can do :

>>> amda.collection.find(var.dataset_id)
Dataset item (id:tao-ura-sw, name:SW / Input OMNI, start:2010-01-01 00:00:00, stop:2021-02-19 00:00:00, n_param:7)h
    Parameter item (id:ura_sw_n, name:density, units:cm⁻³, disp:None, dataset:tao-ura-sw, n:1)
    Parameter item (id:ura_sw_v, name:velocity, units:km/s, disp:None, dataset:tao-ura-sw, n:2)
    Parameter item (id:ura_sw_t, name:temperature, units:eV, disp:None, dataset:tao-ura-sw, n:1)
    Parameter item (id:ura_sw_pdyn, name:dynamic pressure, units:nPa, disp:None, dataset:tao-ura-sw, n:1)
    Parameter item (id:ura_sw_b, name:b tangential, units:nT, disp:None, dataset:tao-ura-sw, n:1)
    Parameter item (id:ura_sw_bx, name:b radial, units:nT, disp:None, dataset:tao-ura-sw, n:1)
    Parameter item (id:ura_sw_da, name:angle Uranus-Sun-Earth, units:deg, disp:None, dataset:tao-ura-sw, n:1)

The object returned by amdapy.amda.Collection.find() can be passed to the amdapy.amda.AMDA.get() method to retrive the contents of the parameter or dataset.

iter_dataset()[source]

Collection item iterator

Returns

collection items

Return type

amdapy.amda.Collection.Item

class amdapy.amda.Dataset(el, data)[source]

Bases: object

AMDA dataset container. Retrieving data from AMDA can be time consuming, as such the object constructor can be called without setting the data.

Parameters
  • el (amdapy.amdaWSClient.client.DatasetElement) – dataset unique identificator

  • data (array type) – optional (default: None), dataset content

Get list of parameter ids :

>>> [p.id for p in dataset.iter_parameter()]
['ura_sw_n', 'ura_sw_v', 'ura_sw_t', 'ura_sw_pdyn', 'ura_sw_b', 'ura_sw_bx', 'ura_sw_da']

You can then get the contents of the parameters by using the bracket operator :

>>> dataset["ura_sw_n"]
iter_parameter()[source]

Parameter iterator

Returns

parameter object

Return type

amdapy.amda.Parameter

Parameter iterator. For example get a list of parameter name and units :

>>> [p.name for p in dataset.iter_parameter()]
['density', 'velocity', 'temperature', 'dynamic pressure', 'b tangential', 'b radial', 'angle Uranus-Sun-Earth']
>>> [p.units for p in dataset.iter_parameter()]
['cm⁻³', 'km/s', 'eV', 'nPa', 'nT', 'nT', 'deg']
make_parameter_list(el)[source]
time()[source]

Get time value for this dataset

Returns

time

Return type

pandas.DataFrame

Get the datasets time values.

>>> T = dataset.time()
>>> type(T)
<class 'pandas.core.indexes.datetimes.DatetimeIndex'>
>>> T
DatetimeIndex(['2010-01-01 01:00:00', '2010-01-01 02:00:00',
               '2010-01-01 03:00:00', '2010-01-01 04:00:00',
...
               '2010-01-01 23:00:00', '2010-01-02 00:00:00'],
              dtype='datetime64[ns]', name='Time', freq=None)
class amdapy.amda.Parameter(id, name, units, data)[source]

Bases: object

Container class for storing parameter objects.

Parameters
  • id (str) – identifier of the parameter, should be unique

  • name (str) – name of the parameter as seen in the AMDA navigation tree

  • units (str) – units in which are expressed the parameters values

  • data (pandas.DataFrame) – values of the parameter

This class is a container for parameter data including identification, name, units and data. We retrieve amdapy.amda.Parameter instances by passing a amdapy.amda.Collection.Parameter object to the amdapy.amda.AMDA.get() or amdapy.amda.AMDA.get_dataset() method.

>>> parameter
Parameter (id:ura_sw_n, name:density, units:cm⁻³, shape: (24, 1))
plot(dataset_id=None, figsize=None)[source]

Plot the parameter. Naive plotting function for visualizing time series data.

Returns

pyplot figure object

Return type

pyplot figure

Module contents