Maxent model for mcat

This page contains some analysis of the Maxent model for mcat, created Sat May 04 09:44:23 CDT 2019 using Maxent version 3.4.1. If you would like to do further analyses, the raw data used here is linked to at the end of this page.

Analysis of omission/commission

The following picture shows the omission rate and predicted area as a function of the cumulative threshold. The omission rate is is calculated both on the training presence records, and (if test data are used) on the test records. The omission rate should be close to the predicted omission, because of the definition of the cumulative threshold.

The next picture is the receiver operating characteristic (ROC) curve for the same data. Note that the specificity is defined using predicted area, rather than true commission (see the paper by Phillips, Anderson and Schapire cited on the help page for discussion of what this means). This implies that the maximum achievable AUC is less than 1. If test data is drawn from the Maxent distribution itself, then the maximum possible test AUC would be 0.903 rather than 1; in practice the test AUC may exceed this bound.

Some common thresholds and corresponding omission rates are as follows. If test data are available, binomial probabilities are calculated exactly if the number of test samples is at most 25, otherwise using a normal approximation to the binomial. These are 1-sided p-values for the null hypothesis that test points are predicted no better than by a random prediction with the same fractional predicted area. The "Balance" threshold minimizes 6 * training omission rate + .04 * cumulative threshold + 1.6 * fractional predicted area.

Cumulative threshold	Cloglog threshold	Description	Fractional predicted area	Training omission rate	Test omission rate	P-value
1.000	0.023	Fixed cumulative value 1	0.489	0.019	0.056	5.024E-5
5.000	0.111	Fixed cumulative value 5	0.329	0.037	0.056	7.586E-8
10.000	0.181	Fixed cumulative value 10	0.252	0.037	0.167	3.744E-7
0.717	0.017	Minimum training presence	0.523	0.000	0.056	1.501E-4
21.456	0.332	10 percentile training presence	0.156	0.093	0.222	8.361E-9
24.691	0.375	Equal training sensitivity and specificity	0.138	0.130	0.222	1.626E-9
15.592	0.258	Maximum training sensitivity plus specificity	0.197	0.037	0.222	1.76E-7
15.177	0.253	Equal test sensitivity and specificity	0.200	0.037	0.222	2.214E-7
7.118	0.142	Maximum test sensitivity plus specificity	0.291	0.037	0.056	9.946E-9
4.053	0.092	Balance training omission, predicted area and threshold value	0.350	0.019	0.056	2.16E-7
10.887	0.194	Equate entropy of thresholded and original distributions	0.241	0.037	0.167	2.071E-7

Pictures of the model

This is the projection of the Maxent model for mcat onto the environmental variables in D:\Workspace\Steamboat\island_head\mxe. Warmer colors show areas with better predicted conditions. White dots show the presence locations used for training, while violet dots show test locations. Click on the image for a full-size version.

Click here to interactively explore this prediction using the Explain tool. If clicking from your browser does not succeed in starting the tool, try running the script in D:\Workspace\Steamboat\island_head\mcat_mxe_explain.bat directly. This tool requires the environmental grids to be small enough that they all fit in memory.

The following picture shows where the prediction is most affected by variables being outside their training range, while projecting the Maxent model onto the environmental variables in D:\Workspace\Steamboat\island_head\mxe. The values shown in the picture give the absolute difference in predictions when using vs not using clamping. (Clamping means that environmental variables and features are restricted to the range of values encountered during training.) Warmer colors show areas where the treatment of variable values outside their training ranges is likely to have a large effect on predicted suitability.

The following two pictures compare the environmental similarity of variables in mxe to the environmental data used for training the model. In the first picture (MESS), areas in red have one or more environmental variables outside the range present in the training data, so predictions in those areas should be treated with strong caution. The second picture (MoD) shows the most dissimilar variable, i.e., the one that is furthest outside its training range. For details, see Elith et al., Methods in Ecology and Evolution, 2010

Response curves

These curves show how each environmental variable affects the Maxent prediction. The curves show how the predicted probability of presence changes as each environmental variable is varied, keeping all other environmental variables at their average sample value. Click on a response curve to see a larger version. Note that the curves can be hard to interpret if you have strongly correlated variables, as the model may depend on the correlations in ways that are not evident in the curves. In other words, the curves show the marginal effect of changing exactly one variable, whereas the model may take advantage of sets of variables changing together.

In contrast to the above marginal response curves, each of the following curves represents a different model, namely, a Maxent model created using only the corresponding variable. These plots reflect the dependence of predicted suitability both on the selected variable and on dependencies induced by correlations between the selected variable and other variables. They may be easier to interpret if there are strong correlations between variables.

Analysis of variable contributions

The following table gives estimates of relative contributions of the environmental variables to the Maxent model. To determine the first estimate, in each iteration of the training algorithm, the increase in regularized gain is added to the contribution of the corresponding variable, or subtracted from it if the change to the absolute value of lambda is negative. For the second estimate, for each environmental variable in turn, the values of that variable on training presence and background data are randomly permuted. The model is reevaluated on the permuted data, and the resulting drop in training AUC is shown in the table, normalized to percentages. As with the variable jackknife, variable contributions should be interpreted with caution when the predictor variables are correlated.

Variable	Percent contribution	Permutation importance
q5_velocity	31.4	4
q5_slope	23	5.7
q95_froude	15.2	4.5
q95_ss	12.2	4.1
q5_froude	8	17.3
q5_ss	5.1	41.1
q95_velocity	2.4	20
q95_slope	1.3	0
q5_depth	1	0.4
q95_depth	0.3	1.8
q95_reynolds	0.1	1
q5_reynolds	0	0

The following picture shows the results of the jackknife test of variable importance. The environmental variable with highest gain when used in isolation is q5_ss, which therefore appears to have the most useful information by itself. The environmental variable that decreases the gain the most when it is omitted is q5_ss, which therefore appears to have the most information that isn't present in the other variables.

The next picture shows the same jackknife test, using test gain instead of training gain. Note that conclusions about which variables are most important can change, now that we're looking at test data.

Lastly, we have the same jackknife test, using AUC on test data.

Raw data outputs and control parameters

The data used in the above analysis is contained in the next links. Please see the Help button for more information on these.
The model applied to the training environmental layers
The model applied to the environmental layers in D:\Workspace\Steamboat\island_head\mxe
The coefficients of the model
The omission and predicted area for varying cumulative and raw thresholds
The prediction strength at the training and (optionally) test presence sites
Results for all species modeled in the same Maxent run, with summary statistics and (optionally) jackknife results

Regularized training gain is 1.440, training AUC is 0.937, unregularized training gain is 1.793.
Unregularized test gain is 1.268.
Test AUC is 0.884, standard deviation is 0.035 (calculated as in DeLong, DeLong & Clarke-Pearson 1988, equation 2).
Algorithm terminated after 500 iterations (22 seconds).

The follow settings were used during the run:
54 presence records used for training, 18 for testing.
10054 points used to determine the Maxent distribution (background points and presence points).
Environmental layers used (all continuous): q5_depth q5_froude q5_reynolds q5_slope q5_ss q5_velocity q95_depth q95_froude q95_reynolds q95_slope q95_ss q95_velocity
Regularization values: linear/quadratic/product: 0.181, categorical: 0.250, threshold: 1.460, hinge: 0.500
Feature types used: hinge linear quadratic
responsecurves: true
jackknife: true
outputfiletype: mxe
outputdirectory: D:\Workspace\Steamboat\island_head
projectionlayers: D:\Workspace\Steamboat\island_head\mxe
samplesfile: D:\Workspace\Steamboat\existing_condition\input_swd\mcat.csv
environmentallayers: D:\Workspace\Steamboat\existing_condition\input_swd\background.csv
randomtestpoints: 25
replicatetype: bootstrap
writeplotdata: true
appendtoresultsfile: true
threads: 10
Command line used:

Command line to repeat this species model: java density.MaxEnt nowarnings noprefixes -E "" -E mcat responsecurves jackknife outputfiletype=mxe outputdirectory=D:\Workspace\Steamboat\island_head projectionlayers=D:\Workspace\Steamboat\island_head\mxe samplesfile=D:\Workspace\Steamboat\existing_condition\input_swd\mcat.csv environmentallayers=D:\Workspace\Steamboat\existing_condition\input_swd\background.csv randomtestpoints=25 replicatetype=bootstrap writeplotdata appendtoresultsfile threads=10