:py:mod:`tape.ensemble_frame` ============================= .. py:module:: tape.ensemble_frame Module Contents --------------- Classes ~~~~~~~ .. autoapisummary:: tape.ensemble_frame.TapeSeries tape.ensemble_frame.TapeFrame tape.ensemble_frame.EnsembleSeries tape.ensemble_frame.EnsembleFrame tape.ensemble_frame.TapeSourceFrame tape.ensemble_frame.TapeObjectFrame tape.ensemble_frame.SourceFrame tape.ensemble_frame.ObjectFrame .. py:class:: TapeSeries(data=None, index=None, dtype: pandas._typing.Dtype | None = None, name=None, copy: bool | None = None, fastpath: bool | pandas._libs.lib.NoDefault = lib.no_default) Bases: :py:obj:`pandas.Series` A barebones extension of a Pandas series to be used for underlying Ensemble data. See https://pandas.pydata.org/docs/development/extending.html#subclassing-pandas-data-structures .. py:property:: _constructor Used when a manipulation result has the same dimensions as the original. .. py:property:: _constructor_sliced .. py:class:: TapeFrame(data=None, index: pandas._typing.Axes | None = None, columns: pandas._typing.Axes | None = None, dtype: pandas._typing.Dtype | None = None, copy: bool | None = None) Bases: :py:obj:`pandas.DataFrame` A barebones extension of a Pandas frame to be used for underlying Ensemble data. See https://pandas.pydata.org/docs/development/extending.html#subclassing-pandas-data-structures .. py:property:: _constructor Used when a manipulation result has the same dimensions as the original. .. py:property:: _constructor_expanddim .. py:class:: EnsembleSeries(expr, label=None, ensemble=None) Bases: :py:obj:`_Frame`, :py:obj:`dask.dataframe.Series` A barebones extension of a Dask Series for Ensemble data. .. py:attribute:: _partition_type .. py:class:: EnsembleFrame(expr, label=None, ensemble=None) Bases: :py:obj:`_Frame`, :py:obj:`dask.dataframe.DataFrame` An extension for a Dask Dataframe for data used by a lightcurve Ensemble. The underlying non-parallel dataframes are TapeFrames and TapeSeries which extend Pandas frames. .. rubric:: Examples Instatiation:: import tape ens = tape.Ensemble() data = {...} # Some data you want tracked by the Ensemble ensemble_frame = tape.EnsembleFrame.from_dict(data, label="my_frame", ensemble=ens) .. py:attribute:: _partition_type .. py:method:: __getitem__(key) .. py:method:: from_tapeframe(data, npartitions=None, chunksize=None, sort=True, label=None, ensemble=None) :classmethod: Returns an EnsembleFrame constructed from a TapeFrame. :param data: Frame containing the underlying data fro the EnsembleFram :type data: `TapeFrame` :param npartitions: The number of partitions of the index to create. Note that depending on the size and index of the dataframe, the output may have fewer partitions than requested. :type npartitions: `int`, optional :param chunksize: Size of the individual chunks of data in non-parallel objects that make up Dask frames. :type chunksize: `int`, optional :param sort: Whether to sort the frame by a default index. :type sort: `bool`, optional :param label: The label used to by the Ensemble to identify the frame. :type label: `str`, optional :param ensemble: A link to the Ensemble object that owns this frame. :type ensemble: `tape.Ensemble`, optional :returns: **result** -- The constructed EnsembleFrame object. :rtype: `tape.EnsembleFrame` .. py:method:: from_dask_dataframe(df, ensemble=None, label=None) :classmethod: Returns an EnsembleFrame constructed from a Dask dataframe. :param df: a Dask dataframe to convert to an EnsembleFrame :type df: `dask.dataframe.DataFrame` or `list` :param ensemble: A link to the Ensemble object that owns this frame. :type ensemble: `tape.ensemble.Ensemble`, optional :param label: The label used to by the Ensemble to identify the frame. :type label: `str`, optional :returns: **result** -- The constructed EnsembleFrame object. :rtype: `tape.EnsembleFrame` .. py:method:: update_ensemble() Updates the Ensemble linked by the `EnsembelFrame.ensemble` property to track this frame. :returns: **result** -- The Ensemble object which tracks this frame, `None` if no such Ensemble. :rtype: `tape.Ensemble` .. py:method:: from_dict(data, npartitions, orient='columns', dtype=None, columns=None, label=None, ensemble=None) :classmethod: Construct a Tape EnsembleFrame from a Python Dictionary :param data: Of the form {field : array-like} or {field : dict}. :type data: dict :param npartitions: The number of partitions of the index to create. Note that depending on the size and index of the dataframe, the output may have fewer partitions than requested. :type npartitions: int :param orient: The "orientation" of the data. If the keys of the passed dict should be the columns of the resulting DataFrame, pass 'columns' (default). Otherwise if the keys should be rows, pass 'index'. If 'tight', assume a dict with keys ['index', 'columns', 'data', 'index_names', 'column_names']. :type orient: {'columns', 'index', 'tight'}, default 'columns' :param dtype: Data type to force, otherwise infer. :type dtype: bool :param columns: Column labels to use when ``orient='index'``. Raises a ValueError if used with ``orient='columns'`` or ``orient='tight'``. :type columns: string, optional :param label: The label used to by the Ensemble to identify the frame. :type label: `str`, optional :param ensemble: A link to the Ensemble object that owns this frame. :type ensemble: `tape.ensemble.Ensemble`, optional :returns: **result** -- The constructed EnsembleFrame object. :rtype: `tape.EnsembleFrame` .. py:method:: from_parquet(path, index=None, columns=None, label=None, ensemble=None, **kwargs) :classmethod: Returns an EnsembleFrame constructed from loading a parquet file. :param path: Source directory for data, or path(s) to individual parquet files. Prefix with a protocol like s3:// to read from alternative filesystems. To read from multiple files you can pass a globstring or a list of paths, with the caveat that they must all have the same protocol. :type path: `str` or `list` :param index: Field name(s) to use as the output frame index. Default is None and index will be inferred from the pandas parquet file metadata, if present. Use False to read all fields as columns. :type index: `str`, `list`, `False`, optional :param columns: Field name(s) to read in as columns in the output. By default all non-index fields will be read (as determined by the pandas parquet metadata, if present). Provide a single field name instead of a list to read in the data as a Series. :type columns: `str` or `list`, optional :param label: The label used to by the Ensemble to identify the frame. :type label: `str`, optional :param ensemble: A link to the Ensemble object that owns this frame. :type ensemble: `tape.ensemble.Ensemble`, optional :returns: **result** -- The constructed EnsembleFrame object. :rtype: `tape.EnsembleFrame` .. py:method:: convert_flux_to_mag(flux_col, zero_point, err_col=None, zp_form='mag', out_col_name=None) Converts this EnsembleFrame's flux column into a magnitude column, returning a new EnsembleFrame. :param flux_col: The name of the EnsembleFrame flux column to convert into magnitudes. :type flux_col: 'str' :param zero_point: The name of the EnsembleFrame column containing the zero point information for column transformation. :type zero_point: 'str' :param err_col: The name of the EnsembleFrame column containing the errors to propagate. Errors are propagated using the following approximation: Err= (2.5/log(10))*(flux_error/flux), which holds mainly when the error in flux is much smaller than the flux. :type err_col: 'str', optional :param zp_form: The form of the zero point column, either "flux" or "magnitude"/"mag". Determines how the zero point (zp) is applied in the conversion. If "flux", then the function is applied as mag=-2.5*log10(flux/zp), or if "magnitude", then mag=-2.5*log10(flux)+zp. :type zp_form: `str`, optional :param out_col_name: The name of the output magnitude column, if None then the output is just the flux column name + "_mag". The error column is also generated as the out_col_name + "_err". :type out_col_name: 'str', optional :returns: **result** -- A new EnsembleFrame object with a new magnitude (and error) column. :rtype: `tape.EnsembleFrame` .. py:method:: coalesce(input_cols, output_col, drop_inputs=False) Combines multiple input columns into a single output column, with values equal to the first non-nan value encountered in the input cols. :param input_cols: The list of column names to coalesce into a single column. :type input_cols: `list` :param output_col: The name of the coalesced output column. :type output_col: `str`, optional :param drop_inputs: Determines whether the input columns are dropped or preserved. If a mapped column is an input and dropped, the output column is automatically assigned to replace that column mapping internally. :type drop_inputs: `bool`, optional :returns: **ensemble** -- An ensemble object. :rtype: `tape.ensemble.Ensemble` .. py:class:: TapeSourceFrame(data=None, index: pandas._typing.Axes | None = None, columns: pandas._typing.Axes | None = None, dtype: pandas._typing.Dtype | None = None, copy: bool | None = None) Bases: :py:obj:`TapeFrame` A barebones extension of a Pandas frame to be used for underlying Ensemble source data See https://pandas.pydata.org/docs/development/extending.html#subclassing-pandas-data-structures .. py:property:: _constructor Used when a manipulation result has the same dimensions as the original. .. py:property:: _constructor_expanddim .. py:class:: TapeObjectFrame(data=None, index: pandas._typing.Axes | None = None, columns: pandas._typing.Axes | None = None, dtype: pandas._typing.Dtype | None = None, copy: bool | None = None) Bases: :py:obj:`TapeFrame` A barebones extension of a Pandas frame to be used for underlying Ensemble object data. See https://pandas.pydata.org/docs/development/extending.html#subclassing-pandas-data-structures .. py:property:: _constructor Used when a manipulation result has the same dimensions as the original. .. py:property:: _constructor_expanddim .. py:class:: SourceFrame(expr, ensemble=None) Bases: :py:obj:`EnsembleFrame` A subclass of EnsembleFrame for Source data. .. py:attribute:: _partition_type .. py:method:: __getitem__(key) .. py:method:: from_parquet(path, index=None, columns=None, ensemble=None) :classmethod: Returns a SourceFrame constructed from loading a parquet file. .. py:method:: from_dask_dataframe(df, ensemble=None) :classmethod: Returns a SourceFrame constructed from a Dask dataframe. :param df: a Dask dataframe to convert to a SourceFrame :type df: `dask.dataframe.DataFrame` or `list` :param ensemble: A link to the Ensemble object that owns this frame. :type ensemble: `tape.ensemble.Ensemble`, optional :returns: **result** -- The constructed SourceFrame object. :rtype: `tape.SourceFrame` .. py:class:: ObjectFrame(expr, ensemble=None) Bases: :py:obj:`EnsembleFrame` A subclass of EnsembleFrame for Object data. .. py:attribute:: _partition_type .. py:method:: from_parquet(path, index=None, columns=None, ensemble=None) :classmethod: Returns an ObjectFrame constructed from loading a parquet file. .. py:method:: from_dask_dataframe(df, ensemble=None) :classmethod: Returns an ObjectFrame constructed from a Dask dataframe. :param df: a Dask dataframe to convert to an ObjectFrame :type df: `dask.dataframe.DataFrame` or `list` :param ensemble: A link to the Ensemble object that owns this frame. :type ensemble: `tape.ensemble.Ensemble`, optional :returns: **result** -- The constructed ObjectFrame object. :rtype: `tape.ObjectFrame`