7. Cards aplication

7.1. Cards.models

class cards.models.CalcStats(*args, **kwargs)

Model for the cards CalcStats.

Functions:

Applies filtering (either Fourier filter of specified order, or Kalman filter with specified gain scale factor) to all geotiff input images for the specified time period. Also performs statistics on the resulting imagery, giving mean, min, max, median values per pixel.

Fields:

output_tile_subdir: Output directory

year_group: Relation with the YearGroup model from the applications GSI

area: Relation with the Area model from the applications GSI

period: Time period over which statistics are calculated

doy_variable: Character field

filter: Order of Fourier series to fit to time-series (default=0)

filter_out: Filtered outputs

input_fourier: Character field

out_dir: Output directory (if non-MODIS tile specified as first argument)

path_spec_location: The path to the location of a special file

class cards.models.CardItem(*args, **kwargs)

Model for the cards CardItem.

Functions:

It writes to the database all models Cards application.

CONTENT_LIMIT:

The list of the all models Cards application.

Fields:

name: Model name

content_type: Relation with the ContentType model

object_id: ID of the element from the model

content_object: Associates model element and its ID

order: The order of an element

class cards.models.Collate(*args, **kwargs)

Model for the cards Collate.

Functions:

Processing GeoTIFF files to specify the tiles.

Fields:

area: Relation with the Area model from the applications GSI

mode: Data are MODIS, DEM and soil

output_tile_subdir: Output tile subdirectory/filename root

input_scale_factor: Input scale factor

input_data_directory: Relation with the InputDataDirectory model from the applications GSI

input_files: Input ad-hoc geotiff filename, assumed to be in $RF_AUXDATA_DIR directory. Relation with the InputDataDirectory model from the applications GSI

class cards.models.MergeCSV(*args, **kwargs)

Model for the cards MergeCSV.

Functions:

Data on csv files.

Fields:

csv1: Filename

csv2: Filename

class cards.models.NamedModel(*args, **kwargs)

Abstract model.

Inherits the “name” field

class cards.models.OrderedCardItem(*args, **kwargs)

Model for the cards OrderedCardItem.

Functions:

It writes to the database all models Cards application.

CONTENT_LIMIT:

The list of the all models Cards application.

Fields:

name: Model name

content_type: Relation with the ContentType model

object_id: ID of the element from the model

content_object: Associates model element and its ID

order: The order of an element the CardItem model

class cards.models.ParallelModel(*args, **kwargs)

Abstract model.

Inherits the “run_parallel” field

class cards.models.PreProc(*args, **kwargs)

Model for the cards PreProc.

Functions:

The normal use for PreProc is to extract selected bands from MODIS hdf files (in the per-tile directory structure under $MODIS_DIR), saving the extracted images as GeoTiffs in the appropriate per-tile location in the directory structure under $SAT_TIF_DIR.

Fields:

area: Relation with the Area model from the applications GSI

mode: Extracts GeoTiffs from hdf and performs stats for selected year(s)

year_group: Relation with the YearGroup model from the applications GSI

path_spec_location: The full path to a specific hdf file

class cards.models.QRF(*args, **kwargs)

Model for the cards QRF.

Functions:

The Quantile Random Forest (QRF) process analyses all of the decision trees generated by RFtrain, looking at the distributions of all Y-parameter values across all trees, to determine the confidence that can be placed on each predicted value; this information can be used subsequently in scoring, to provide a range of statiscs for each scored pixel, including: Median, lower quartile, upper quartile, min (eg <5%) quantile, max (eg >95%) quantile, standard error, standard deviation, variance, and per-pixel quantile value.

Fields:

interval: Confidence interval (overrides $QRF_INTERVAL)

number_of_trees: Number of trees to use for QRF (overrides actual number of trees created)

number_of_threads: Number of parallel threads to use for QRF

directory: Identifies parent directory for trees, if not in default location (under $RF_DIR)

class cards.models.RFScore(*args, **kwargs)

Model for the cards RFScore.

Functions:

RFscore uses the Random Forest decision trees generated by RFtrain to predict the Y- parameter value for each combination of X-parameter values in the specified input files. The range of possible output files depends on whether QRF has been run, or not. If QRF has been run for the selected training set, then a number of statistical measures can be output, (based on the setting of the <QRFopts> command line argument). If QRF has not been run, then the only output will be the Conditional Mean of the predicted Y-parameters values.

Fields:

area: Relation with the Area model from the applications GSI

year_group: Relation with the YearGroup model from the applications GSI

bias_corrn:Nnumber of parallel threads to use for QRF

number_of_threads: Number of parallel threads used for QRF, if enabled ($QRF_INTERVAL>0)

QRFopts: Controls outputs: 0=None, 1=Mean, 2=Median, 4=Min, 8=Max, 16=LowerQ, 32=UpperQ, 64=QRFstats (set OR of values), 128=Std Error, 256=Std Dev, 512=Variance (sigma sq), 1024=nVar, 2048=per pixel Quantile

ref_target: Datatype used for training (eg ‘Biomass’), which triggers generation of new target <90th quantile

clean_name: Text to uniquely identify filename for new cleaned target (<RefTarget>_<CleanName>)

class cards.models.RFTrain(*args, **kwargs)

Model for the cards RFTrain.

Functions:

Core Random Forest processing, which uses the specified “target” (Y-parameter) imagery as the basis for training using all the specified input “X-parameter” images as the basis. (this will enable subsequent scoring using other images (different times/locations) for X- parameters to derive the target Y-parameter values for those times and locations.

The main output from RFtrain are a set of decision trees, which define how any combination of input X-parameters can be used to predict a corresponding output Y-parameter.

Fields:

tile_type: Relation with the TileType model from the applications GSI

number_of_trees: The number of trees to use in randomForest (default = 50)

value: Value

config_file: The configuration file

output_tile_subdir: The output tile subdirectory

input_scale_factor: The input scale factor

training: Integer field

number_of_variable: Number of iterations used for tree node (or automatically calculated if not defined)

number_of_thread: Number of parallel threads used for tree generation (default is one, if not defined)

class cards.models.RandomForest(*args, **kwargs)

Model for the cards RandomForest.

Functions:

Random Forest processing.

Fields:

aoi_name: Character field

satellite: Relation with the Satellite model from the applications GSI

param_set: Text field

run_set: Character field

model: Character field

mvrf: Character field

class cards.models.Remap(*args, **kwargs)

Model for the cards Remap.

Functions:

Extracts/mosaics and remaps the specified input image(s) to the specified output min/max lat/long limits. If a shape file is specified, then the polygons within the shape file are used to identify which portions of the output will have remapped pixel values (all regions outside the polygons will be set to NO_DATA (=-9999)). This utility can be used for remapping files stored within the MODIS tiles directory structure (for a specified parameter name), with output as WGS84 image. It can also be used for extracting a subset of an existing WGS84 image and rescaling to the same size, with the same spatial resolution as other extracted images. The latter functionality can alternatively be achieved using the GDAL utility – gdalwarp. Remap also provides the option to output images as .png format files, using a user-specified colour look-up table (LUT). The lut may be in any of three formats: ”.lut” as used by the OpenSource ImageJ program, .pal format, which are simple ASCII text files, with 2 header lines, then each of 256 lines containing “R G B” values, or .txt files which allow for specifying the LUT in terms of breakpoints, where each line contain 4 values (“breakpoint R G B”), and the software interpolates between each breakpoint to give the final 256 colour range.

Fields:

year_group: Relation with the YearGroup model from the applications GSI

file_spec: <subdir>/<fileroot> below $RF_DIR/Tiles/hxxvyy

roi: <filename> With single line comprising: MinLat MinLon MaxLat MaxLon

model_name: The name of the model

output_root: Pathname and root for filename of all outputs generated (<OutRoot>_<ParamName>.tif)

output_suffix: Suffix for output filenames, eg <OutRoout>/<ParamName>_<OutSuffix>.tif

scale: Spatial resolution of required output (1000=1km, 250=250m etc)

output: 1 for .tif, 2 for .png, 4 for .csv (OR value for multiple outputs)

color_table: Full pathname for required colour LUT for .png

refstats_file: Geotiff <filename>.tif

refstats_scale: Scale factor for RefStatsFile pixel values

conditional_mean: Boolean field

conditional_min: Boolean field

conditional_median: Boolean field

conditional_max: Boolean field

lower_quartile: Boolean field

upper_quartile: Boolean field

class cards.models.YearFilter(*args, **kwargs)

Model for the cards YearFilter.

Functions:

Applies filtering across multiple years (either Fourier filter of specified order, or Kalman filter with specified gain scale factor) to all geotiff input images for the specified time period, covering (per-year) output from Calcstats or output from RFscore.

Fields:

area: Relation with the Area model from the applications GSI

filetype: Values to determine which files to output

filter: Smoothing filter method

filter_output: Selection of output images

extend_start: Number of years to extend the start backwards to attempt better early fit (default=0)

input_fourier: Value for NDVI files, maximum Fourier order used in CalcStats

output_directory: Specific output directory

input_directory: Specific input directory

cards.models.auto_add_card_item(sender, instance=None, created=False, **kwargs)

To write a new element models of the Cards applications in the CardItem model.

cards.models.get_card_item(self)

Get an element the CardItem model or create a new.

7.2. Cards.views

cards.views.proces_card_new_run(request, *args, **kwargs)

View at the creation of new cards.

When choosing a new card in “processing_card_menu” function transferred dictionary “rev_url” values:  {<button_a_new_card>: <url_to_create_a_new_card>}

cards.views.processing_card_menu(request, rev_url, args=False)

The function to receive a response depending on the type of card.

Arguments:

request: The request is sent to the server when processing the page

rev_url: Dictionary. It contains values for reverse after processing of form.

rev_url[<name_card>][0]: url for reverse,

If the element to edit, then the variable is its id:

rev_url[<name_card>][1]: id of element,

args: Boolean. If the value is True, then edit the element and contains id

7.3. Cards.views_card_runid_csid

cards.views_card_runid_csid.runid_csid_calcstats_add(request, *args, **kwargs)

View for to create a new CalcStats card.

Arguments:

• run_id: ID of the Run
• cs_id: ID of the CardSequence

cards.views_card_runid_csid.runid_csid_calcstats_edit(request, *args, **kwargs)

View for to edit the CalcStats card.

Arguments:

• run_id: ID of the Run
• cs_id: ID of the CardSequence

cards.views_card_runid_csid.runid_csid_collate_add(request, *args, **kwargs)

View for to create a new Collate card.

Arguments:

• run_id: ID of the Run
• cs_id: ID of the CardSequence

cards.views_card_runid_csid.runid_csid_collate_edit(request, *args, **kwargs)

View for to edit the Collate card.

Arguments:

• run_id: ID of the Run
• cs_id: ID of the CardSequence

cards.views_card_runid_csid.runid_csid_mergecsv_add(request, *args, **kwargs)

View for to create a new MergeCSV card.

Arguments:

• run_id: ID of the Run
• cs_id: ID of the CardSequence

cards.views_card_runid_csid.runid_csid_mergecsv_edit(request, *args, **kwargs)

View for to edit the MergeCSV card.

Arguments:

• run_id: ID of the Run
• cs_id: ID of the CardSequence

cards.views_card_runid_csid.runid_csid_preproc_add(request, *args, **kwargs)

View for to create a new PreProc card.

Arguments:

• run_id: ID of the Run
• cs_id: ID of the CardSequence

cards.views_card_runid_csid.runid_csid_preproc_edit(request, *args, **kwargs)

View for to edit the PreProc card.

Arguments:

• run_id: ID of the Run
• cs_id: ID of the CardSequence

cards.views_card_runid_csid.runid_csid_qrf_add(request, *args, **kwargs)

View for to create a new QRF card.

Arguments:

• run_id: ID of the Run
• cs_id: ID of the CardSequence

cards.views_card_runid_csid.runid_csid_qrf_edit(request, *args, **kwargs)

View for to edit the QRF card.

Arguments:

• run_id: ID of the Run
• cs_id: ID of the CardSequence

cards.views_card_runid_csid.runid_csid_randomforest_add(request, *args, **kwargs)

View for to create a new RandomForest card.

Arguments:

• run_id: ID of the Run
• cs_id: ID of the CardSequence

cards.views_card_runid_csid.runid_csid_randomforest_edit(request, *args, **kwargs)

View for to edit the RandomForest card.

Arguments:

• run_id: ID of the Run
• cs_id: ID of the CardSequence

cards.views_card_runid_csid.runid_csid_remap_add(request, *args, **kwargs)

View for to create a new Remap card.

Arguments:

• run_id: ID of the Run
• cs_id: ID of the CardSequence

cards.views_card_runid_csid.runid_csid_remap_edit(request, *args, **kwargs)

View for to edit the Remap card.

Arguments:

• run_id: ID of the Run
• cs_id: ID of the CardSequence

cards.views_card_runid_csid.runid_csid_rfscore_add(request, *args, **kwargs)

View for to create a new RFScore card.

Arguments:

• run_id: ID of the Run
• cs_id: ID of the CardSequence

cards.views_card_runid_csid.runid_csid_rfscore_edit(request, *args, **kwargs)

View for to edit the RFScore card.

Arguments:

• run_id: ID of the Run
• cs_id: ID of the CardSequence

cards.views_card_runid_csid.runid_csid_rftrain_add(request, *args, **kwargs)

View for to create a new RFTrain card.

Arguments:

• run_id: ID of the Run
• cs_id: ID of the CardSequence

cards.views_card_runid_csid.runid_csid_rftrain_edit(request, *args, **kwargs)

View for to edit the RFTrain card.

Arguments:

• run_id: ID of the Run
• cs_id: ID of the CardSequence

cards.views_card_runid_csid.runid_csid_year_filter_add(request, *args, **kwargs)

View for to create a new YearFilter card.

Arguments:

• run_id: ID of the Run
• cs_id: ID of the CardSequence

cards.views_card_runid_csid.runid_csid_year_filter_edit(request, *args, **kwargs)

View for to edit the YearFilter card.

Arguments:

• run_id: ID of the Run
• cs_id: ID of the CardSequence

7.4. Cards.cards_forms

This module contains methods for creating and editing card forms.

class cards.cards_forms.CalcStatsForm(*args, **kwargs)

Form for editing CalcStats Card.

class cards.cards_forms.CollateForm(*args, **kwargs)

Form for editing Collate Card.

class cards.cards_forms.MergeCSVForm(*args, **kwargs)

Form for editing MergeCSV Card.

class cards.cards_forms.PreProcForm(*args, **kwargs)

Form for editing PreProc Card.

class cards.cards_forms.QRFForm(*args, **kwargs)

Form for editing QRF Card.

class cards.cards_forms.RFScoreForm(*args, **kwargs)

form for editing RFScore Card.

class cards.cards_forms.RFTrainForm(*args, **kwargs)

Form for editing RFTrain Card.

class cards.cards_forms.RandomForestForm(*args, **kwargs)

Form for editing RandomForest Card.

class cards.cards_forms.RemapForm(*args, **kwargs)

Form for editing Remap Card.

class cards.cards_forms.YearFilterForm(*args, **kwargs)

Form for editing YearFilter Card.

7.5. Cards.card_update_create

This module contains methods to refresh the data in the cards after you create and edit.

cards.card_update_create.calcstats_update_create(form, item_id=None)

Updated card CalcStats.

Arguments:

• form: Object of the form
• item_id: ID of the card. Set when editing (the default=None when you create a card)

cards.card_update_create.collate_update_create(form, item_id=None, multiple=None, delete=False)

Updated card Collate.

Arguments:

• form: Object of the form
• item_id: ID of the card. Set when editing (the default=None when you create a card)

cards.card_update_create.mergecsv_update_create(form, item_id=None)

Updated card MergeCSV.

Arguments:

• form: Object of the form
• item_id: ID of the card. Set when editing (the default=None when you create a card)

cards.card_update_create.preproc_update_create(form, item_id=None)

Updated card PreProc.

Arguments:

• form: Object of the form
• item_id: ID of the card. Set when editing (the default=None when you create a card)

cards.card_update_create.qrf_update_create(form, item_id=None)

Updated card QRF.

Arguments:

• form: Object of the form
• item_id: ID of the card. Set when editing (the default=None when you create a card)

cards.card_update_create.randomforest_update_create(form, item_id=None)

Updated card RandomForest.

Arguments:

• form: Object of the form
• item_id: ID of the card. Set when editing (the default=None when you create a card)

cards.card_update_create.remap_update_create(form, item_id=None)

Updated card Remap.

Arguments:

• form: Object of the form
• item_id: ID of the card. Set when editing (the default=None when you create a card)

cards.card_update_create.rfscore_update_create(form, item_id=None)

Updated card RFScore.

Arguments:

• form: Object of the form
• item_id: ID of the card. Set when editing (the default=None when you create a card)

cards.card_update_create.rftrain_update_create(form, item_id=None)

Updated card RFTrain.

Arguments:

• form: Object of the form
• item_id: ID of the card. Set when editing (the default=None when you create a card)

cards.card_update_create.year_filter_update_create(form, item_id=None)

Updated card YearFilter.

Arguments:

• form: Object of the form
• item_id: ID of the card. Set when editing (the default=None when you create a card)