Documentation

This is an affiliated package for the AstroPy package. The documentation for this package is here:

PyDL

Introduction

This is the documentation for PyDL.

Components

Most of the functionality of PyDL is in sub-packages.

Goddard Utilities (pydl.goddard)

Introduction

This is a placeholder for Goddard Utilities documentation.

API

pydl.goddard Package

This subpackage contains the Goddard utilities.

pydl.goddard.astro Module

This module corresponds to the goddard/astro directory in idlutils.

Functions

airtovac(air)	Convert air wavelengths to wavelengths in vacuum.
gcirc(ra1, dec1, ra2, dec2[, units])	Computes rigorous great circle arc distances.
get_juldate([seconds])	Returns the current Julian date.
get_juldate_main()	Entry point for the get_juldate command-line script.
vactoair(vacuum)	Convert vacuum wavelengths to wavelengths in air.

pydl.goddard.math Module

This module corresponds to the goddard/math directory in idlutils.

Functions

flegendre(x, m)

Compute the first m Legendre polynomials.

pydl.goddard.misc Module

This module corresponds to the goddard/misc directory in idlutils.

Functions

cirrange(ang[, radians])

Convert an angle larger than 360 degrees to one less than 360 degrees.

SDSS Imaging Data (pydl.photoop)

Introduction

This is a placeholder for photoop documentation.

API

pydl.photoop Package

This subpackage implements functions from the photoop package.

Classes

PhotoopException

Exceptions raised by pydl.photoop that don’t fit into a standard exception class like ValueError.

Class Inheritance Diagram

pydl.photoop.photoobj Module

This module corresponds to the photoobj directory in photoop.

Functions

sdss_calibv()	Return calibration for velocities from pix/frame to deg/day.
unwrap_objid(objid)	Unwrap CAS-style objID into run, camcol, field, id, rerun.

pydl.photoop.sdssio Module

This module corresponds to the sdssio directory of photoop.

Functions

filtername(f)	Return the name of a filter given its number.
filternum([filt])	Return index number for SDSS filters either from a number or name.
sdss_calib(run, camcol, field[, rerun])	Read photometric calibration solutions from calibPhotom or calibPhotomGlobal files.
sdss_name(ftype, run, camcol, field[, …])	Return the name of an SDSS data file including path.
sdss_path(ftype, run[, camcol, rerun])	Return the path name for SDSS data assuming SAS directory structure.
sdssflux2ab(flux[, magnitude, ivar])	Convert the SDSS calibrated fluxes (magnitudes) into AB fluxes (magnitudes).

pydl.photoop.window Module

This module corresponds to the window directory in photoop.

Functions

sdss_score(flist[, silent])	Score a list of imaging fields from zero to one.
window_read(**kwargs)	Read window files in $PHOTO_RESOLVE.
window_score(**kwargs)	For uber-resolve, score all the fields from zero to one.

SDSS Spectroscopic Data (pydl.pydlspec2d)

Introduction

This is a placeholder for pydlspec2d documentation.

API

pydl.pydlspec2d Package

This subpackage implements functions from the idlspec2d package.

Classes

Pydlspec2dException	Exceptions raised by pydl.pydlspec2d that don’t fit into a standard exception class like ValueError.
Pydlspec2dUserWarning	Class for warnings issued by pydl.pydlspec2d.

Class Inheritance Diagram

pydl.pydlspec2d.spec1d Module

This module corresponds to the spec1d directory in idlspec2d.

Functions

findspec(*args, **kwargs)	Find SDSS/BOSS spectra that match a given RA, Dec.
latest_mjd(plate, **kwargs)	Find the most recent MJD associated with a plate.
number_of_fibers(plate, **kwargs)	Returns the total number of fibers per plate.
pca_solve(newflux, newivar[, maxiter, …])	Replacement for idlspec2d pca_solve.pro.
plot_eig(filename[, title])	Plot spectra from an eigenspectra/template file.
preprocess_spectra(flux, ivar[, loglam, …])	Handle the processing of input spectra through the combine1fiber() stage.
readspec(platein[, mjd, fiber])	Read SDSS/BOSS spec2d & spec1d files.
skymask(invvar, andmask[, ormask, ngrow])	Mask regions where sky-subtraction errors are expected to dominate.
spec_append(spec1, spec2[, pixshift])	Append the array spec2 to the array spec1 & return a new array.
spec_path(plate[, path, topdir, run2d])	Return the directory containing spPlate files.
template_input(inputfile, dumpfile[, flux, …])	Collect spectra and pass them to PCA or HMF solvers to compute spectral templates.
template_input_main()	Entry point for the compute_templates script.
template_metadata(inputfile[, verbose])	Read template metadata from file.
template_qso(metadata, newflux, newivar[, …])	Run PCA or HMF on QSO spectra.
template_star(metadata, newloglam, newflux, …)	Run PCA or HMF on stellar spectra of various classes.
wavevector(minfullwave, maxfullwave[, …])	Return an array of wavelengths.

Classes

HMF(spectra, invvar[, K, n_iter, seed, …])

Class used to manage data for Heteroscedastic Matrix Factorization (HMF).

Class Inheritance Diagram

pydl.pydlspec2d.spec2d Module

This module corresponds to the spec2d directory in idlspec2d.

Functions

aesthetics(flux, invvar[, method])	Add nice values to a spectrum where it is masked.
combine1fiber(inloglam, objflux, newloglam)	Combine several spectra of the same object, or resample a single spectrum.
filter_thru(flux[, waveimg, wset, mask, …])	Compute throughput in SDSS filters.

SDSS Utilities (pydl.pydlutils)

Introduction

This package provides functionality similar to idlutils, a general suite of tools heavily used by SDSS.

idlutils is itself divided into a number of subpackages. Below we list the subpackages and the usability of the PyDL equivalent. The readiness levels are defined as:

Not Applicable (NA)

Another built-in or numpy/scipy/astropy package completely replaces this. No point in implementing.

None

Not (yet) implemented at all.

Rudimentary

Only a few functions are implemented.

Fair

Enough functions are implemented to be useful, but some are missing.

Good

Pretty much anything you could do with the idlutils code you can do with the equivalent here.

Subpackage	Readiness Level	Comments
2mass	None	For use with matching 2MASS catalogs to SDSS data.
astrom	None	For use with SDSS astrometric data structures. Largely superseded by WCS.
bspline	Good
cooling	Good
coord	Fair	Some functionality already provided by astropy.coordinates.
cosmography	None
dimage	None
djsphot	None
dust	None	For use with the SFD galactic dust map.
first	None	For use with matching FIRST catalogs to SDSS data.
fits	NA	Use astropy.io.fits.
healpix	None
image	Rudimentary	Only a few functions implemented.
json	NA	Use json or other packages.
mangle	Fair	Some work still required on polygon area calculations.
math	Fair
mcmc	None	But there are plenty of good Python MCMC packages out there.
mglib	None
misc	Fair
mpeg	None
mpfit	None
physics	None
plot	None	Much functionality already exists in matplotlib.
psf	None
rgbcolor	Good
rosat	None	For use with matching ROSAT catalogs to SDSS data.
sdss	Good	Most important functionalities are bitmasks and reading sweep files.
slatec	None
spheregroup	Good	Used for matching arbitrary RA, Dec coordinates to other arbitrary RA, Dec coordinates.
TeXtoIDL	NA	This package is for including TeX in IDL plots. Since matplotlib understands TeX natively, this is not needed.
trace	Fair
ukidss	None	Used for matching UKIDSS catalogs to SDSS data.
wise	None	Used for matching WISE catalogs to SDSS data.
yanny	Good

API

pydl.pydlutils Package

This subpackage implements functions from the idlutils package.

Classes

PydlutilsException	Exceptions raised by pydl.pydlutils that don’t fit into a standard exception class like ValueError.
PydlutilsUserWarning	Class for warnings issued by pydl.pydlutils.

Class Inheritance Diagram

pydl.pydlutils.bspline Module

This module corresponds to the bspline directory in idlutils.

Functions

cholesky_band(l[, mininf, verbose])	Compute Cholesky decomposition of banded matrix.
cholesky_solve(a, bb)	Solve the equation Ax=b where A is a Cholesky-banded matrix.
iterfit(xdata, ydata[, invvar, upper, …])	Iteratively fit a b-spline set to data, with rejection.

Classes

bspline(x[, nord, npoly, bkpt, bkspread, …])

Bspline class.

Class Inheritance Diagram

pydl.pydlutils.cooling Module

This module corresponds to the cooling directory in idlutils.

Functions

read_ds_cooling(fname[, logT])

Read in the Sutherland & Dopita (1993) cooling function.

pydl.pydlutils.coord Module

This module corresponds to the coord directory in idlutils.

Functions

current_mjd()	Return the current MJD.
munu_to_radec(munu, icrs_frame)	Convert from SDSS great circle coordinates to equatorial coordinates.
radec_to_munu(icrs_frame, munu)	Convert from equatorial coordinates to SDSS great circle coordinates.
stripe_to_eta(stripe)	Convert from SDSS great circle coordinates to equatorial coordinates.
stripe_to_incl(stripe)	Convert from SDSS stripe number to an inclination relative to the equator.

Classes

SDSSMuNu(*args, **kwargs)

SDSS Great Circle Coordinates

Class Inheritance Diagram

pydl.pydlutils.image Module

This module corresponds to the image directory in idlutils.

Functions

djs_maskinterp(yval, mask[, xval, axis, const])	Interpolate over masked pixels in a vector, image or 3-D array.
djs_maskinterp1(yval, mask[, xval, const])	Interpolate over a masked, 1-d array.

pydl.pydlutils.mangle Module

This module corresponds to the mangle directory in idlutils.

Mangle [1] is a software suite that supports the concept of polygons on a sphere, and is used to, for example, describe the window function of the Sloan Digital Sky Survey.

This implementation is intended to support the portions of Mangle that are included in idlutils. To simplify the coding somewhat, unlike idlutils, the caps information is accessed through polygon.x and polygon.cm, not polygon.caps.x or polygon.caps.cm.

Window function operations are already supported by is_in_polygon() and is_in_window(). However, calculation of polygon area (solid angle) from first principles is not yet implemented.

Note that in traditional geometry “spherical polygon” means a figure bounded by great circles. Mangle allows polygons to be bounded by any circle, great or not. Use care when comparing formulas in this module to formulas in the mathematical literature.

References

[1]	Swanson, M. E. C.; Tegmark, Max; Hamilton, Andrew J. S.; Hill, J. Colin, 2008 MNRAS 387, 1391.

Functions

angles_to_x(points[, latitude])	Convert spherical angles to unit Cartesian vectors.
cap_distance(x, cm, points)	Compute the distance from a point to a cap, and also determine whether the point is inside or outside the cap.
circle_cap(radius, points)	Construct caps based on input points (directions on the unit sphere).
is_cap_used(use_caps, i)	Returns True if a cap is used.
is_in_cap(x, cm, points)	Are the points in a (single) given cap?
is_in_polygon(polygon, points[, ncaps])	Are the points in a given (single) polygon?
is_in_window(polygons, points[, ncaps])	Check to see if points lie within a set of polygons.
read_fits_polygons(filename[, convert])	Read a “polygon” format FITS file.
read_mangle_polygons(filename)	Read a “polygon” format ASCII file in Mangle’s own format.
set_use_caps(polygon, index_list[, add, …])	Set the bits in use_caps for a polygon.
x_to_angles(points[, latitude])	Convert unit Cartesian vectors to spherical angles.

Classes

FITS_polygon	Handle polygons read in from a FITS file.
ManglePolygon(*args, **kwargs)	Simple object to represent a polygon.
PolygonList(*args, **kwargs)	A list that contains ManglePolygon objects and possibly some metadata.

Class Inheritance Diagram

pydl.pydlutils.math Module

This module corresponds to the math directory in idlutils.

Functions

djs_median(array[, dimension, width, boundary])	Compute the median of an array.
djs_reject(data, model[, outmask, inmask, …])	Routine to reject points when doing an iterative fit to data.
find_contiguous(x)	Find the longest sequence of contiguous non-zero array elements.

Classes

computechi2(bvec, sqivar, amatrix)

Solve the linear set of equations \(A x = b\) using SVD.

Class Inheritance Diagram

pydl.pydlutils.misc Module

This module corresponds to the misc directory in idlutils.

Functions

decode_mixed(x)	Convert bytes in Numpy arrays into strings.
djs_laxisgen(dims[, iaxis])	Returns an integer array where each element of the array is set equal to its index number along the specified axis.
djs_laxisnum(dims[, iaxis])	Returns an integer array where each element of the array is set equal to its index number in the specified axis.
hogg_iau_name(ra, dec[, prefix, precision])	Properly format astronomical source names to the IAU convention.
hogg_iau_name_main()
struct_print(array[, filename, formatcodes, …])	Print a NumPy record array (analogous to an IDL structure) in a nice way.

pydl.pydlutils.rgbcolor Module

This module corresponds to the rgbcolor directory in idlutils.

This code reproduces the algorithms of Lupton et al. (2004) [2]. The purpose is to produce nice color (RGB, JPEG, PNG, etc.) from FITS images in astronomical filter bands.

References

[2]	Lupton, Robert, et al., 2004 PASP 116, 113.

Functions

nw_arcsinh(colors[, nonlinearity])	Scales colors by a degree of nonlinearity specified by user.
nw_cut_to_box(colors[, origin])	Limits the pixel values of the image to a ‘box’, so that the colors do not saturate to white but to a specific color.
nw_float_to_byte(image[, bits])	Converts an array of floats in [0.0, 1.0] to bytes in [0, 255].
nw_scale_rgb(colors[, scales])	Multiply RGB image by color-dependent scale factor.

pydl.pydlutils.sdss Module

This module corresponds to the sdss directory in idlutils.

Functions

default_skyversion()	Returns skyversion number to use for photoop imaging.
sdss_astrombad(run, camcol, field[, …])	For a list of RUN, CAMCOL, FIELD, return whether each field has bad astrometry.
sdss_flagexist(flagname, bitname[, …])	Check for the existence of flags.
sdss_flagname(flagname, flagvalue[, concat])	Return a list of flag names corresponding to the values.
sdss_flagval(flagname, bitname)	Convert bitmask names into values.
sdss_objid(run, camcol, field, objnum[, …])	Convert SDSS photometric identifiers into CAS-style ObjID.
sdss_specobjid(plate, fiber, mjd, run2d[, …])	Convert SDSS spectrum identifiers into CAS-style specObjID.
sdss_sweep_circle(ra, dec, radius[, stype, …])	Read the SDSS datasweep files and return objects around a location.
set_maskbits([idlutils_version, maskbits_file])	Populate the maskbits cache.
unwrap_specobjid(specObjID[, run2d_integer, …])	Unwrap CAS-style specObjID into plate, fiber, mjd, run2d.

pydl.pydlutils.spheregroup Module

This module corresponds to the spheregroup directory in idlutils.

Functions

spheregroup(ra, dec, linklength[, chunksize])	Perform friends-of-friends grouping given ra/dec coordinates.
spherematch(ra1, dec1, ra2, dec2, matchlength)	Match points on a sphere.

Classes

chunks(ra, dec, minSize)	chunks class
groups(coordinates, distance[, separation])	Group a set of objects (a list of coordinates in some space) based on

Class Inheritance Diagram

pydl.pydlutils.trace Module

This module corresponds to the trace directory in idlutils.

Functions

fchebyshev(x, m)	Compute the first m Chebyshev polynomials.
fchebyshev_split(x, m)	Compute the first m Chebyshev polynomials, but modified to allow a split in the baseline at \(x=0\).
fpoly(x, m)	Compute the first m simple polynomials.
func_fit(x, y, ncoeff[, invvar, …])	Fit x, y positions to a functional form.
traceset2xy(tset[, xpos, ignore_jump])	Convert from a trace set to an array of x,y positions.
xy2traceset(xpos, ypos, **kwargs)	Convert from x,y positions to a trace set.

Classes

TraceSet(*args, **kwargs)

Implements the idea of a trace set.

Class Inheritance Diagram

pydl.pydlutils.yanny Module

This module corresponds to the yanny directory in idlutils.

This is a Python library for reading & writing yanny files.

yanny is an object-oriented interface to SDSS Parameter files (a.k.a. “FTCL” or “yanny” files) following these specifications. Parameter files typically have and can be recognized by the extension .par. These files may also be recognized if the first line of the file is:

#%yanny

This is not part of the standard specification, but it suffices to identify, e.g., files that have not yet been written to disk, but only exist as file objects.

Because Parameter files can contain multiple tables, as well as metadata, there is no simple, one-to-one correspondence between these files and, say, an astropy Table object. Thus, yanny objects are implemented as a subclass of OrderedDict (to remember the order of keyword-value pairs), and the actual data are values accessed by keyword. Still, it is certainly possible to write a table-like object to a yanny file.

Given the caveats above, we have introduced experimental support for reading and writing yanny files directly to/from Table objects. Because of the experimental nature of this support, it must be activated “by hand” by including this snippet in your code:

from astropy.table import Table
from astropy.io.registry import (register_identifier, register_reader,
                                 register_writer)
from pydl.pydlutils.yanny import (is_yanny, read_table_yanny,
                                  write_table_yanny)

register_identifier('yanny', Table, is_yanny)
register_reader('yanny', Table, read_table_yanny)
register_writer('yanny', Table, write_table_yanny)

Currently multidimensional arrays are only supported for type char, and a close reading of the specifications indicates that multidimensional arrays were only ever intended to be supported for type char. So no multidimensional arrays, sorry.

Functions

is_yanny(origin, path, fileobj, *args, **kwargs)	Identifies Yanny files or objects.
read_table_yanny(filename[, tablename])	Read a yanny file into a Table.
write_ndarray_to_yanny(filename, datatables)	Converts a NumPy record array into a new FTCL/yanny file.
write_table_yanny(table, filename[, …])	Write a Table to a yanny file.

Classes

yanny([filename, raw])

An object interface to a yanny file.

Class Inheritance Diagram

Other Notes

PyDL Changelog

0.6.0 (2017-09-19)

• This release is compatible with Astropy 2.0, and may be backwards incompatible with astropy v1.x.
• Update to astropy_helpers/v2.0.1.
• Use standard library argparse (Issue #31).
• Use the new astropy.coordinates.Attribute class.
• Fix typo (PR #26).

0.5.4 (2017-05-04)

• Added sdss_specobjid() to compute SDSS specObjIDs, and its inverse function unwrap_specobjid().
• Update to astropy_helpers/v1.3.1.
• Refactor HMF code into an object to contain the data and methods.
• Use functions from astropy.utils.data where possible.
• Fix an integer division error encountered when using Numpy 1.12 (Issue #19).
• Fixed tests that were failing on 32-bit platforms and Python 3.5 (Issue #20).

0.5.3 (2016-12-03)

• Fixed formatting of TODO document.
• Fixed tests that were failing on 32-bit platforms (Issue #14).
• Use temporary files so that tests can run when astropy is installed read-only (e.g., with pip; Issue #16)

0.5.2 (2016-08-04)

• Changes in how Mangle-polygon containing FITS files are handled, related to Issue #11.
• Fixed memory leak in combine1fiber(), see Issue #12.
• Added is_in_window().
• Allow polygon area functions to deal with negative caps and use_caps.
• Update docs/conf.py for Python 3.5 compatibility (PR #13).

0.5.1 (2016-06-22)

• Removed unnecessary from __future__ import in pydl.pydlspec2d.spec1d.
• Ongoing documentation upgrades.
• Update some links that needed to be transitioned from SDSS-III to SDSS-IV.
• Upgrade to astropy_helpers/v1.2.
• Update to latest version of package-template.
• Disabled tests on Python 3.3; enabled tests on Python 3.5
• Fix Issue #8; Issue #9.
• Add warnings about incomplete Mangle functions.

0.5.0 (2016-05-01)

• Dropped support for Python 2.6. Python 2.6 does not contain collections.OrderedDict, which is needed to support yanny objects, and at this point it is not worth going to the trouble to support this with an external package.
• Ongoing review and upgrade of docstrings.
• Yanny files can now be converted into genuine NumPy record arrays; previously, the conversion was only to numpy.ndarray with named columns, which is a slightly different thing.
• Added additional tests on yanny objects.
• Experimental support for interconversion of yanny objects and Table objects.
• Improving PEP 8 compliance
• Restructuing sub-packages to reduce the number of files.
• Improvements to spectral template processing code, deduplicated some code.
• Support platform-independent home directory (PR #7).
• Uppercase the package name (in documentation only).
• Upgrade to astropy_helpers/v1.1.1.
• Add functions from the idlutils rgbcolor directory.
• spec_path() can now find SDSS spectra, not just BOSS.

0.4.1 (2015-09-22)

• No changes at all from 0.4.0. This tag only exists because of a botched PyPI upload.

0.4.0 (2015-09-22)

• Use astropy_helpers/v1.0.3, package-template/v1.0.
• Remove some old FITS code that astropy.io.fits makes moot.
• Remove code for command-line scripts. These are now auto-generated by the “entry_point” method.
• Remove Python/3.2 tests.
• Improved test coverage.
• Fixed problem with the spheregroup code.
• Removed some code that is 100% redundant with astropy (e.g. angles_to_xyz()).
• Fixed bug in set_use_caps() that was discovered on the IDL side.
• Updated documentation of read_fits_polygons().
• Added cross-references to classes, functions, etc.

0.3.0 (2015-02-20)

• Use astropy_helpers/v0.4.3, package-template/v0.4.1.
• Avoided (but did not fix) a bug in chunks that occurs when operating on a list of coordinates of length 1.
• Fixed a typo in bspline, added documentation.
• Simplify documentation files.
• sdss_flagname() now accepts more types of numeric input.
• Added Authors and Credits file.

0.2.3 (2014-07-22)

• Added pydl.photoop.window.
• Added stub sdss_calib(), updated sdss_score().
• Added unwrap_objid().
• Merged pull request #4, fixing some Python3 issues.

0.2.2 (2014-05-07)

• Updated to latest package-template version.
• Added ability to write multiple ndarray to yanny files.
• Fixed struct_print() test for older Numpy versions.
• Fixed failing yanny file test.
• Improve test infrastructure, including Travis builds.
• Allow comment characters inside quoted strings in yanny files.

0.2.1 (2013-10-06)

• Added sdss_sweep_circle().
• Added first few pydl.photoop functions.
• Clean up some import statements.

0.2.0 (2013-04-22)

• Using the astropy package-template to bring pydl into astropy-compatible form.
• Some but not all tests are re-implemented.

0.1.1 (2013-03-06)

• Creating a tag representing the state immediately after creation of the git repository.

0.1 (2010-11-10)

• Initial tag (made in svn, not visible in git). Visible at http://www.sdss3.org/svn/repo/pydl/tags/0.1 .

TODO

• Increase test coverage.
• pydl.pydlutils.mangle needs more work.
  • Area (solid angle) calculation.
  • Area calculation needs to account for the use_caps attribute.
  • Intersection of caps with other caps.

Authors and Credits

Thank you to everyone who has contributed to PyDL. If we have neglected to include you, please let us know right away!

• Kyle Barbary
• Larry Bradley
• Matt Craig
• Christoph Deil
• Mike DiPompeo
• igoldste
• Duncan Macleod
• Demitri Muna
• Thomas Robitaille
• Brigitta Sipocz
• Erik Tollerud
• ViviCoder
• Benjamin Alan Weaver
• Kyle Westfall

Base API

pydl Package

Python replacements for functions that are part of the IDL built-in library, or part of astronomical IDL libraries. The emphasis is on reproducing results of the astronomical library functions. Only the bare minimum of IDL built-in functions are implemented to support this.

Functions

file_lines(path[, compress])	Replicates the IDL FILE_LINES() function.
median(array[, width, axis, even])	Replicate the IDL MEDIAN() function.
rebin(x, d[, sample])	Resize x to new dimensions given by d.
smooth(signal, owidth[, edge_truncate])	Replicates the IDL SMOOTH() function.
uniq(x[, index])	Replicates the IDL UNIQ() function.

Classes

pcomp(x[, standardize, covariance])	Replicates the IDL PCOMP() function.
PydlException	Base class for exceptions raised in PyDL functions.

Note

Do not edit this page - instead, place all documentation for the affiliated package inside packagename/