Low-level API¶

Carton¶

class target_selection.cartons.base.BaseCarton(targeting_plan, config_file=None, schema=None, table_name=None)[source]¶

Bases: object

A base class for target cartons.

This class is not intended for direct instantiation. Instead, it must be subclassed and the relevant class attributes overridden with the values corresponding to the carton.

Parameters:

targeting_plan (str) – The target selection plan version.
config_file (str) – The path to the configuration file to use. If undefined, uses the internal target_selection.yml file.
schema (str) – Schema in which the temporary table with the results of the query will be created. If None, tries to use the schema parameter from the configuration file for this plan of target selection. If the parameter is not set, defaults to 'sandbox'.
table_name (str) – The name of the temporary table. Defaults to temp_<name> where <name> is the target class name.

Variables:

name (str) – The name of the carton (required).
cadence (str) – The label of the cadence rule for this carton.
category (str) – The category of targets for this carton.
mapper (str) – The mapper with which this carton is associated.
orm (str) – The ORM library to be used, peewee or sqlalchemy.
tag (str) – The version of the target_selection code used.
query_region (tuple) – A tuple defining the region over which the query should be performed, with the format (ra, dec, radius) in degrees. This will append a q3c_radial_query condition to the query.
load_magnitudes (bool) – Whether to load target magnitudes. In general this must be True except for cartons for which it’s known the magnitudes will not be used, e.g., skies.

add_optical_magnitudes()[source]¶: Adds gri magnitude columns.

abstract build_query(version_id, query_region=None)[source]¶

Builds and returns the query.

The ORM query for the target class. Note that this must be the un-executed query, which will be executed in run. The select statement must be run on catalogdb.catalog and needs to include, at least, catalogid. Additional columns such as ra, dec, pmra, pmdec can be specified but are otherwise completed when calling run. Magnitude columns can be included and propagated to targetdb but must be aliased as magnitude_<band> where <band> must be one of the columns in targetdb.magnitude.

The query returned must include a filter for version_id on catalogdb.catalog. Normally this is applied to the Catalog select as .where(Catalog.version_id == version_id).

Parameters:: version_id (int) – The id of the cross-match version to use.
Returns:: query – A Select or ModelSelect query.

check_targets()[source]¶: Check if data has been loaded for this carton and targeting plan.

drop_carton()[source]¶: Drops the entry in targetdb.carton.

drop_table()[source]¶: Drops the intermediate table if it exists.

get_model()[source]¶: Returns a Peewee model for the temporary table using reflection.

get_version_id()[source]¶: Returns the version_id for the cross-match plan.

load(mode='fail', overwrite=False)[source]¶

Loads the output of the intermediate table into targetdb.

Parameters:

mode (str) – The mode to use when loading the targets. If 'fail', raises an error if the carton already exist. If 'overwrite', overwrites the targets. If 'append', appends the targets.
overwrite (bool) – Equivalent to setting mode='overwrite'. This option is deprecated and will raise a warning.

post_process(model, **kwargs)[source]¶

Post-processes the temporary table.

This method provides a framework for applying non-SQL operations on carton query. It receives the model for the temporary table and can perform any operation on it, including modifying the selected column with a mask of targets to be used.

This method can also be used to set the cadence column in the temporary table. This column will be used to set the target cadence if the carton cadence attribute is not set.

post_process runs inside a database transaction so it’s not necessary to create a new one, but savepoints can be added.

Parameters:: model (Model) – The model of the intermediate table.
Returns:: mask (tuple) – The list of catalogids from the temporary table that should be selected as part of this carton. If True (the default), selects all the records.

run(query_region=None, overwrite=False, limit=None, add_optical_magnitudes=True, **post_process_kawrgs)[source]¶

Executes the query and post-process steps, and stores the results.

This method calls build_query and runs the returned query. The output of the query is stored in a temporary table whose schema and table name are defined when the object is instantiated.

After the query has run, the post_process routine is called if the method has been overridden for the given carton.

Parameters:

query_region (tuple) – A tuple defining the region over which the query should be performed, with the format (ra, dec, radius) in degrees. This will append a q3c_radial_query condition to the query.
overwrite (bool) – Whether to overwrite the intermediary table if already exists.
limit (int or None) – Limit the query to this number of targets. Useful for testing. The LIMIT statement is added to the query returned by build_query and the exact behaviour will depend on the query.
post_process_args (dict) – Keyword arguments to be passed to post_process.

Returns:

model (Model) – The model for the intermediate table.

setup_transaction()[source]¶

Setups the transaction locally modifying the datbase parameters.

This method runs inside a transaction and can be overridden to set the parameters of the transaction manually. It applies to both run and load.

write_table(filename=None, mode='results', write=True)[source]¶

Writes the selection to a FITS file.

Parameters:

filename (str) – The file to which to write the table. Defaults to <name>_<plan>.fits.
mode (str) – Defines what data to write. If 'results', writes the intermediate table (usually just the catalogid column). If 'targetdb', writes all the relevant columns for the targets loaded to targetdb for this carton and plan (must be used after load has been called).
write (bool) – Whether to write the table to disk. If False, just returns the table object.

Returns:

table (Table) – A table object with the selected results.

cadence = None¶

can_offset = False¶

category = None¶

instrument = None¶

load_magnitudes = True¶

mapper = None¶

name = None¶

property path¶: The schema-qualified path to the output table.

priority = None¶

program = None¶

query_region = None¶

value = None¶

target_selection.cartons.tools.get_file_carton(filename)[source]¶: Returns a carton class that creates a carton based on a FITS file. The FITS file is located in the open_fiber_path which is specified in python/config/target_selection.yml. The list of FITS files to be loaded is specified in the file open_fiber_file_list.txt which is in the directory open_fiber_path.

XMatch¶

class target_selection.xmatch.XMatchPlanner(database, models, plan, run_id, version_id=None, extra_nodes=[], order='hierarchical', key='row_count', epoch=2016.0, start_node=None, query_radius=None, schema='catalogdb', temp_schema='sandbox', output_table='catalog', log=None, log_path='./xmatch_{plan}.log', debug=False, show_sql=False, sample_region=None, database_options=None, path_mode='full', join_paths=None)[source]¶

Prepares and runs catalogue cross-matching.

This class prepares the execution of a cross-matching between multiple catalogues, the result being an output table of unique targets linked to each input catalog via a relational table. Target coordinates are propagated to a common epoch when proper motions are available. Instantiating the class only prepares the process and sets up the processing order; cross-matching itself happens by calling the run method.

The output table contains a sequential integer identifier for each unique target (catalogid), along with the following columns: ra, dec, pmra, pmdec, parallax, lead, and version_id. version_id relates the record in the version table which contains the cross-matching plan and the tag of the code used when it was run. lead indicates from which one of the input catalogues the coordinates were obtained.

The output table is related to each input catalogue via a many-to-many table with the format <output_table>_to_<catalogue_table> where <output_table> is the table name of the output table and <catalog_table> is the table name of the referred catalogue. Each relational table contains the unique identifier column, catalogid, target_id pointing to the primary key of the referred catalogue, version_id, and a boolean best indicating whether it is the best (closest) possible match when a target in the output table correspond to multiple targets in the input catalogue.

The cross-matching process roughly follows the following process:

In practice the cross-matching process begins by creating a graph of all nodes (tables to be processed and additional tables, extra_nodes, in the database) and edges (foreign keys relating two tables). This graph is used to determine join conditions and to establish the order in which the input models will be processed. The processing order is determined by the order and key input parameters. When key='row_count', tables are sorted by number of decreasing rows so that tables with more targets are processed first (note that to speed things up the row count is always the latest known approximate determined by ANALYZE); if key='resolution' the associated spatial resolution for a catalogue is used to process catalogues with high resolution first. If order='hierarchical', all the tables are divided into as many disconnected subgraphs as exist; then for each subgraph the maximum row count or minim resolution is calculated (depending on the value of key). Subgraphs are sorted based on this result and then tables belonging to each subgraph are sorted by key. If order='global' the key ordering is applied to all tables without taking into account subgraphs.

To speed things up unique targets are initially inserted into a temporary table <output_table>_<uid> where <output_table> is the name of the output table and <uid> is a unique identifier based on the version.

Once the order has been determined and when run is called, each table model is processed in order. The first model is just ingested completely into the temporary table and its associated relational table is created if it does not exist (the equivalent of phase 3 below).

For each additional model the following three stages are applied:

In phase 1 we determine what targets in the input model have an existing cross-match to targets already ingested into the temporary table. To do that we build all possible joins between the model and the temporary table. If multiple joins are possible via a given table only the shortest is used (see get_join_paths). For all matched targets we insert entries in the relational table. The lead of the original entries is not changed.
In phase 2 we perform the actual cross-match between targets in the temporary table and the ones in the input catalogue. Currently the only cross-matching method available is a spatial cone query with radius query_radius. All matched targets are added to the relational table and the one with the smallest distance is defined as best.
In phase 3 we determine any target in the input catalogue that has not been cross-matched at this point and insert them into the temporary table as new entries. The lead is set to the input catalogue and the one-to-one match is added to the relational table.

In phases 1 and 3 the queries are initially stored as a Postgresql temporary table for efficiency, and then copied to the relational table. After all the tables have been processed the output temporary table is inserted in bulk into the output table and dropped.

In addition to the limitations of the spatial cone query method, the following caveats are known:

Input tables with duplicate targets are not currently supported.
In phase 2 there is no current measure in place for the same target to be associated with more than one catalogid (i.e., each cross-match is performed independently).

Parameters:

database (PostgresqlDatabase) – A PostgresqlDatabase to the database the tables to cross-match.
models (list) – The list of XMatchModel classes to be cross-matched. If the model correspond to a non-existing table it will be silently ignored.
plan (str) – The cross-matching plan version.
run_id (int) – An integer to identify this run of cross-matching. The ID is bit shifted RUN_ID_BIT_SHIFT positions and added to the catalogid. This allows to quickly associate a catalogid with a run without having to query catalogdb. A run_id cannot be used if there are targets already in catalog using that same run_id.
version_id – The catalogdb.version.id cross-match version to use. Normally this will be None, in which case a new version id will be created. For “addendum” runs, this should be set to the run to which to append.
extra_nodes (list) – List of PeeWee models to be used as extra nodes for joins (i.e., already established cross-matches between catalogues). This models are not processed or inserted into the output table.
order (str or list) – The type of sorting to be applies to the input models to decide in what order to process them. Currently allowed values are 'hierarchical' and 'global' (refer to the description above). The order can also be a list of table names, in which case that order is used without any further sorting.
key (str) – The key to be used while sorting. Can be 'row_count' or 'resolution'.
epoch (float) – The epoch to which to convert all the target coordinates as they are inserted into the output table.
start_node (str) – If specified, the name of the table that will be inserted first regarding of the above sorting process.
query_radius (float) – The radius, in arcsec, for cross-matching between existing targets. Used in phase 2. Defaults to 1 arcsec.
schema (str) – The schema in which all the tables to cross-match live (multiple schemas are not supported), and the schema in which the output tables will be created.
output_table (str) – The name of the output table. Defaults to catalog.
temp_schema – The schema where the temporary catalog table will be initially created.
log – A logger to which to log messages. If not provided the target_selection logger is used.
log_path (str) – The path to which to log or False to disable file logging.
debug (bool or int) – Controls the level to which to log to the screen. If False no logging is done to stdout. If True the logging level is set to debug (all messages). It’s also possible specify a numerical value for the logging level.
show_sql (bool) – Whether to log the full SQL queries being run.
sample_region (tuple) – Allows to specify a 3-element tuple with the (ra, dec, radius) of the region to which to limit the cross-match. All values must be in degrees. It can also be a list of tuples, in which case the union of all the regions will be sampled.
path_mode (str) – The mode to look for join paths to link tables to output catalog. original mode uses a scorched earth algorithm that iteratively looks the shortest path and removes the first node trasspased and starts again until no shortest paths are available. full mode retrieves all the paths that are not a subsample of another path. config_list mode takes the list of paths from join_paths parameter in the .yml configuration file.
join_paths (list) – When using path_mode=``config_list`` is the list of paths to link tables to output catalog table in phase_1.
database_options (dict) –
A dictionary of database configuration parameters to be set locally during each phase transaction, temporarily overriding the default database configuration. Keys must be the database parameter to modify. The value can be a simple string with the value to set, or a dictionary that more accurately defines when the parameter will be applied. For example
```
database_options:
    work_mem: '2GB'
    temp_buffers : {value: '500MB', phases: [3]}
```
In this case the temp_buffers='500MB' option will only be set for phase 3. Configuration options to be used for a specific table can be set up when defining the XMatchModel.

get_join_paths(source, return_models=False, mode='full')[source]¶

Determines all possible join path between two tables.

Mode original follows a scorched earth approach in which once an edge has been used for a join it cannot be used again. This produces only distinct joins between the two nodes. Mode full includes all possible paths that are not a subsample of another path. In both modes paths that include only the source and destination through their relational table are ignored, as are paths in which the last node before the output table has not yet been processed. Finally mode “config_list” is used to take the paths from a list indicated in the configuration file.

Weights can be defined by setting the join_weight value in XMatchModel. The weight for each edge is the average of the the join_weight of the two nodes joined. The weight defaults to 1. Lower weights translate to better chances of that join path to be selected.

Parameters:

source (str) – The initial table for the path.
return_models (bool) – If True, returns each path as a list of models. Otherwise returns the table names.
mode (str) – The method used to obtain the paths. Possible values are original, full, and config_list

Returns:

list – A list in which each item is a path with the table names or models joining source to dest. The list is sorted in order of increasing path length.

get_output_model(temporary=False)[source]¶: Returns the temporary or final output model (catalog).

get_relational_model(model, sandboxed=False, temp=False, create=False)[source]¶

Gets or creates a relational table for a given model.

When the relational model is sandboxed, the table is created in the temporary schema and suffixed with the same MD5 used for the run.

load_output_tables(model, keep_temp=False)[source]¶: Loads the temporary tables into the output tables.

process_model(model, force=False)[source]¶: Processes a model, loading it into the output table.

classmethod read(in_models, plan, config_file=None, **kwargs)[source]¶

Instantiates XMatchPlanner from a configuration file.

The YAML configuration file must organised by plan string (multiple plans can live in the same file). Any parameter that XMatchPlanner accepts can be passed via the configuration file. Additionally, the configuration file accepts the two extra parameters: exclude, a list of table names that will be ignored (this is useful if you pass a datbase or base class as in_models and want to ignore some of the models), and tables, a dictionary of table names with parameters to be passed to XMatchModel for its corresponding model. An example of a valid configuration files is:

'0.1.0':
    order: hierarchical
    key: resolution
    query_radius: 1.
    schema: catalogdb
    output_table: catalog
    start_node: tic_v8
    debug: true
    log_path: false
    exclude: ['catwise']
    tables:
        tic_v8:
            ra_column: ra
            dec_column: dec
            pmra_column: pmra
            pmdec_column: pmdec
            is_pmra_cos: true
            parallax_column: plx
            epoch: 2015.5
        gaia_dr2_source:
            ra_column: ra
            dec_column: dec
            pmra_column: pmra
            pmdec_column: pmdec
            is_pmra_cos: true
            parallax_column: parallax
            epoch: 2015.5
            skip: true

It is also possible to use a parameter base_plan pointing to a previous plan string. In that case the previous plan configuration will be used as base and the new values will be merged (the update happens recursively as with normal Python dictionaries).

Note that only models that match the table names in tables will be passed to XMatchPlanner to be processed; any other table will be used as an extra joining node unless it’s listed in exclude, in which case it will be ignored completely. It’s possible to set the skip option for a table; this has the same effect as removing the entry in table.

Parameters:

in_models – The models to cross-match. Can be a list or tuple of PeeWee Model instances, a base class from which all the models to use subclass, or a PeeweeDatabaseConnection to the database containing the models. In the latter case, the models must have been imported so that they are available via the models attribute.
plan (str) – The cross-matching plan.
config_file (str or dict) – The path to the configuration file to use. Defaults to config/xmatch.yml. The file must contain a hash with the cross-match plan.
kwargs (dict) – User arguments that will override the configuration file values.

run(vacuum=False, analyze=False, from_=None, force=False, dry_run=False, keep_temp=False)[source]¶

Runs the cross-matching process.

Parameters:

vacuum (bool) – Vacuum all output tables before processing new catalogues.
analyze (bool) – Analyze all output tables before processing new catalogues.
from (str) – Table from which to start running the process. Useful in reruns to skip tables already processed.
force (bool) – Allows to continue even if the temporary table exists or the output table contains records for this version. force=True is assumed if from_ is defined.
dry_run (bool) – If False, loads the temporary tables into Catalog and CatalogToXXX. True implies keep_temp=True; all the cross-matching steps will be run but the original tables won’t be modified. A dry run can only be executed for a plan with a single catalogue since processing multiple catalogue requires the final tables to have been updated for successive catalogues.
keep_temp (bool) – Whether to keep the temporary table or to drop it after the cross matching is done.

set_process_order(order='hierarchical', key='row_count', start_node=None)[source]¶

Sets and returns the order in which tables will be processed.

See XMatchPlanner for details on how the order is decided depending on the input parameters.

show_join_paths()[source]¶

Prints all the available joint paths.

This is useful before call run to make sure the join paths to be used are correct or adjust the table weights otherwise. Note that the paths starting from the first model to be processed are ignored.

update_model_graph()[source]¶: Updates the model graph using models as nodes and fks as edges.

target_selection.xmatch.XMatchModel(Model, resolution=None, ra_column=None, dec_column=None, pmra_column=None, pmdec_column=None, is_pmra_cos=True, parallax_column=None, epoch_column=None, epoch=None, epoch_format='jyear', relational_table=None, has_duplicates=False, has_missing_coordinates=False, skip=False, skip_phases=None, query_radius=None, join_weight=1, database_options=None)[source]¶

Expands the model Metadata with cross-matching parameters.

The parameters defined can be accessed with the same name as Model._meta.xmatch.<parameter> (e.g., Model._meta.xmatch.has_duplicates).

Parameters:

resolution (float) – The spatial resolution of the catalogue, in arcsec.
ra_column (str) – The name of the right ascension column. If not provided, an attempt will be made to recover it from the Q3C index, if it exists. Assumed to be in degrees.
dec_column (str) – As ra, for the declination column.
pmra_column (str) – The RA proper motion column, assumed to be in milliarcseconds per year.
pmdec_column (str) – As pmra_column for the declination proper motion.
is_pmra_cos (bool) – Whether pmra_column provides the RA proper motion corrected from declination(pmra * cos(dec)) or not.
parallax_column (str) – The column containing the parallax, assumed to be in arcsec.
epoch_column (str) – The column containing the epoch of the target coordinates.
epoch (float) – The epoch of the targets that applies to all the records in the table. epoch and epoch_column are mutually exclusive. If neither epoch_column nor epoch are defined, assumes that the epoch is 2015.5.
epoch_format (str) – The format of the epoch. Either Julian year ('jyear') or Julian date ('jd').
table_name (str) – Overrides the default model table name. This can be useful sometimes if, for example, a view has been created that contains only the columns from the main table needed for cross-matching.
has_duplicates (bool) – Whether the table contains duplicates.
has_missing_coordinates (bool) – Whether the catalogue contains rows in which the RA/Dec are null.
skip (bool) – If True, the table will be used as a join node but will not be cross-matched. This is useful for testing and also if the table is in a previous version of the configuration file and you are using the base_version option but want to remove that table. It can also be used when setting a join_path for a model but otherwise don’t want the table to be processed.
skip_phases (list) – A list of cross-matching phases to be skipped for this model. Refer to the XMatchPlanner documentation for definitions on what each phase does.
query_radius (float) – The radius, in arcsec, to use in the radial query for cross-matching. If not provided defaults to the XMatchPlanner value.
join_weight (float) – The weight used by XMatchPlanner.get_join_paths to determine the cost of using this table as a join. Lower weights translate to better chances of that join path to be selected.
database_options (dict) – A dictionary of database configuration parameters to be set locally for this model for each processing phase transaction, temporarily overriding the default database configuration. Keys must be the database parameter to modify. The value can be a simple string with the value to set, or a dictionary that more accurately defines when the parameter will be applied. See XMatchPlanner for more information.

Returns:

Model – The same input model with the additional cross-matching parameters added to the metadata namespace xmatch.

target_selection.xmatch.RUN_ID_BIT_SHIFT = 53¶: Reserve last 11 bits for the run id.

Skies¶

target_selection.skies.get_sky_table(database, table, output, tiles=None, tile_nside=32, candidate_nside=32768, min_separation=10, ra_column='ra', dec_column='dec', mag_column=None, is_flux=False, radius_column=None, flux_unit='nMgy', scale_a=0.2, scale_b=1.0, mag_threshold=None, calculate_min_separation=True, nsample=2048, downsample_data=None, downsample_nside=256, n_cpus=1, seed=None)[source]¶

Identifies skies from a database table.

Skies are selected using the following procedure:

The sky is divided in HEALPix “tiles” of nside tile_nside. For each tile the catalogue is queried to retrieve all the targets that lie in the tile footprint. This assumes that the healpix_ang2ipix_nest Postgresql function from pg_healpix is available. It’s recommended that an index is created for the tiling norder to speed the query.
Each tile is subsequently divided in pixels of nside candidate_nside. Pixels that contain a catalogue target are removed as possible sky candidates. For each target with magnitude less than mag_threshold all pixels within a min_separation are rejected. The remaining pixels are considered valid skies. Alternatively, if mag_column and mag_threshold are defined, the minimum separation to valid pixels is corrected using the expression \(s^* = s + \dfrac{(m_{thr}-m)^{\beta}}{a}\) where \(s\) is the minimum separation, \(m_{thr}\) is the magnitude threshold, \(a=0.2\) and \(\beta=1.0\) are factors that control the relationship between the star’s magnitude and the exclusion radius.
If nsample, only that number of skies are returned for each tile. The tile is divided in pixels of nside downsample_nside (which must be smaller than candidate_nside but larger than tile_nside) and for each downsample pixel int(downsample / downsample_npix) + 1 skies are selected. If not enough valid positions can be selected in this way, each downsample pixel quota is completed with the invalid positions that have a larger separation to their nearest neighbour.
The process can be parallelised for each tile and the results are compiled in a single Pandas data frame that is saved to an HDF5 file.

All pixel values use the nested ordering.

Parameters:

database (PeeweeDatabaseConnection) – A valid database connection.
table (str) – Name of the table to query, optionally schema-qualified.
output (str) – Path to the HDF5 file to write.
tiles (list) – A list of HEALPix pixels of nside tile_nside for which the sky selection will be done. If None, runs for all the pixels of nside tile_nside.
tile_nside (int) – The HEALPix nside to use to tile the all-sky catalogue.
candidate_nside (int) – The HEALPix nside used to identify candidate pixels in each tile. Candidates are then checked to confirm that their closest neighbour is at least min_separation arcsec away.
min_separation (int) – The minimum separation, in arcsec, between skies and their closest neighbour in the catalogue.
ra_column (str) – The name of the column in table that contains the Right Ascension coordinates, in degrees.
dec_column (str) – The name of the column in table that contains the Declination coordinates, in degrees.
mag_column (str) – The name of the column in table with the magnitude to be used to scale min_separation.
is_flux (bool) – If True, assumes the mag_column values are given as fluxes (units of flux_unit).
flux_unit (str) – Gives the units of flux in the ‘mag_column’ - known values ‘nMgy’, ‘Jy’
mag_threshold (float) – The value below which the separation to neighbouring sources will be scaled.
radius_column (str) – Name of the database column that provided the object radius (an alternative to mag_column useful for extended sources)
scale_a (float) – Value of \(a\) in the radius vs mag relationship
scale_b (float) – Value of \(\beta\) in the radius vs mag relationship
calculate_min_separation (bool) – If True, calculates the separation to the nearest neighbour for each candidate sky position.
nsample (int or None) – The total number of skies to retrieve for each tile. If None, returns all candidate skies.
downsample_nside (int) – The HEALPix nside used for downsampling. If nside, the resulting valid skies will be grouped by HEALPix pixels of this resolution. For each pixel a random sample will be drawn so that the total number of skies selected matches nsample.
downsample_data (pandas.DataFrame) – A data frame with previously selected skies that will be used to downsample the sky candidates. This is useful when trying to create a sky catalogue from multiple tables to ensure that the selected sky positions match across the various tables. If not enough valid skies can be selected from the sample in the data frame, they will be completed up to nsample.
n_cpus (int) – Number of CPUs to use for multiprocessing.
seed (int) – The random state seed.

Returns:

skies (pandas.DataFrame) – The list of selected skies.

target_selection.skies.nested_regrade(pixels, nside_in, nside_out)[source]¶

Returns the parent/children pixels from a given HealPix nested pixel.

The HealPix nested mode follows a quadrilateral tree pixel scheme (see Figure 1 in Górski et al. 2005) When the resolution (nside) increases, each pixel is divided in four pixels. The numbering of such pixels follows a simple binary representation achieved by appending binary digits to the new pixels. So, for example, pixel 22 (b10110) becomes four new pixels with binary numbers b1011000, b1011001, b1011010, and b1011011, corresponding to decimal 91, 92, 93, and 94.

This function returns the child pixels from a given pixel when going from resolution nside_in to nside_out, if nside_out > nside_in, or the parent pixel if nside_out < nside_in.

Note that this function works only for pixels using the nested indexing and should not be used with the ring indexing.

Parameters:

pixels (int or ndarray) – The pixels for which we want to get the parents/children. Can be a single integer or an array of indices.
nside_in (int) – The nside of the input pixels. Must be one of \(2^k\) where \(k\in[0,1,2,\dots]\).
nside_out (int) – The destination nside.

Returns:

output (int or ndarray) – If nside_out < nside_in and pixels is a single value, this will be an integer with the parent of the input pixel. If pixels is an array the output will be an array of the same size in which each element is the parent of the corresponding pixel in the input array. If nside_out > nside_in and pixels is an integer, the output will be an array with the child pixels. The size of the array will be \(2^{2l}\) where l=k_out-k_in (\(n_{side}=2^k\)). If the input in an array of pixels, the output will be a 2D array in which each row contains the child pixels of the corresponding pixel in the input array.

target_selection.skies.plot_skies(file_or_data, ra, dec, radius=1.5, targets=None, show_sky_buffer=False, buffer_radius=10.0)[source]¶

Plots the skies (and optionally targets) in a regions.

Parameters:

file_or_data (str or DataFrame) – A HDF5 file with the sky catalogue or a Pandas data frame with the data, including columns labelled ra and dec.
ra (float) – The right ascension of the centre of the field.
dec (float) – The declination of the centre of the field.
radius (float) – The FOV radius, in degrees.
targets (list) – A target list or array as (ra, dec).
show_sky_buffer (bool) – Plots the buffer regions around each sky.
buffer_radius (float) – The sky buffer radius, in arcsec.

Returns:

figure (Figure) – The Matplotlib figure with the plot.

target_selection.skies.plot_sky_density(file_or_data, nside, pix_column=None, nside_plot=32, **kwargs)[source]¶

Plots the number of skies as a HEALPix map.

Parameters:

file_or_data (str or DataFrame) – A HDF5 file with the sky catalogue or a Pandas data frame with the data. It is assumed that the file or data frame contains at least a column pix_X, where X is the order corresponding to nside.
nside (int) – The HEALPix nside for the skies.
pix_column (str or None) – The column that contains the HEALPix pixels. If None, the index of the data frame will be used.
nside_plot (int) – The HEALPix nside in which the data will be plotted.
kwargs (dict) – Other keyword parameters to pass to healpy’s mollview.

Returns:

figure (Figure) – The Matplotlib figure with the plot.

Utils¶

class target_selection.utils.Timer[source]¶

Convenience context manager to time events.

Modified from https://bit.ly/3ebdp3y.

target_selection.utils.copy_pandas(df, database, table_name, schema=None, columns=None)[source]¶

Inserts a Pandas data frame using COPY.

Parameters:

df (DataFrame) – The Pandas data frame to copy or the inputs to be used to create one.
database – A database connection.
table_name (str) – The name of the table into which to copy the data.
schema (str) – The schema in which the table lives.
columns (list) – The list of column names to copy.

target_selection.utils.get_configuration_values(database, parameters)[source]¶: Returns a dictionary of datbase configuration parameter to value.

target_selection.utils.get_epoch(xmodel)[source]¶: Returns the epoch for an XMatchModel in Julian years.

target_selection.utils.remove_version(database, plan, schema='catalogdb', table='catalog', delete_version=True, vacuum=True)[source]¶: Removes all rows in table and table_to_ that match a version.

target_selection.utils.set_config_parameter(database, parameter, new_value, reset=True, log=None)[source]¶: Temporarily a database configuration parameter.

target_selection.utils.sql_apply_pm(ra_field, dec_field, pmra_field, pmdec_field, epoch_delta, is_pmra_cos=True)[source]¶

Constructs a SQL expression for applying proper motions to RA/Dec.

Parameters:

ra_field – The Peewee field or value representing Right Ascension.
dec_field – The Peewee field or value representing Declination.
pmra_field – The Peewee field or value representing the proper motion in RA.
pmdec_field – The Peewee field or value representing the proper motion in Dec.
epoch_delta – A value or expression with the delta epoch, in years.
is_pmra_cos (bool) – Whether the pmra_field includes the correction for the cosine of the declination

Returns:

ra_dec (Expression) – A tuple with the Expression instances for the proper motion corrected RA and Dec.

target_selection.utils.sql_iauname(ra_field, dec_field, prefix='SDSS J')[source]¶

Constructs a SQL expression for the IAU name from RA/Dec.

Parameters:

ra_field – The Peewee field or value representing Right Ascension.
dec_field – The Peewee field or value representing Declination.
prefix (str) – The prefix to add before the calculated IAU name.

Returns:

iauname (Expression) – The Expression instance to construct the IAU name.

target_selection.utils.vacuum_outputs(database, vacuum=True, analyze=True, schema='catalogdb', table='catalog', relational_tables=True)[source]¶: Vacuums and analyses the output tables.

target_selection.utils.vacuum_table(database, table_name, vacuum=True, analyze=True, maintenance_work_mem='50GB')[source]¶: Vacuums and analyses a table.