Save time with ArcGIS and Python

I usually try to avoid using ArcGIS. As I prefer to do geo-spatial analysis in R, using some great packages including sp, raster and rgdal.

But some recent analyses have required some tendonitis inducing* mouse clicking in ArcGIS. Only a simple task of converting rasters to float files. But none the less, thumb, forefinger and mind numbing after a while.

Here is a basic loop which, will convert all rasters in a folder (inFolder) to floats in a output folder (outFolder). You could potentially use this for many different functions in arcpy.

# Import system modules
import arcpy
from arcpy import env

# Set local variables
inFolder = "T:\\Your_Directory\\Environmental Data\\Bioclim_ascii\\Rasters"
outFolder = "T:\\Your_Directory\\Environmental Data\\Bioclim_ascii\\Float"
arcpy.env.workspace = InFolder

for Ras in arcpy.ListRasters():
    arcpy.RasterToFloat_conversion(inFolder + "\\" + Ras, outFolder + "\\" + Ras + ".flt")

* no tendonitis was developed during this demo.

Posted in Uncategorized | Leave a comment

EWC14 – Match Two: Catenaccio & Ecological Niche Theory.

In football, like science, competing paradigms and co-existing concepts are common. Despite the revolutionary ideas and performances of Sebes and Puskás, other tactical systems flourished in the 1950s and 1960s. The Catenaccio (literally meaning door-bolt in Italian) is a tactical system in football which implies a highly organized, structured and competitive backline defence. The tactic focuses on protecting space to minimise goal-scoring opportunities. Making a team defensively compact, while trying to score goals at the other end.

Italy using Catenaccio to stop Spain.

Italy using Catenaccio to stop Spain.

Parallels can be drawn between the Catenaccio system and Ecological Niche theory. Niche theory was formally defined in 1957, when G. Evelyn Hutchinson defined a niche as an n-dimensional hyper-volume. Where the dimensions are environmental conditions and the resources that define the requirements of an individual or a species.

If we think of a football team as species and a player as an individual, we can start to conceptualise niche in practice. Without a competing team (inter-specific competition), a football team is free to pass the ball and have shots at goal. Really, they are just limited by their own abilities and the confines of a football pitch. This is the Fundamental Niche. However, in a match situation opposing teams (species) and players (individuals) are interacting. They compete for the ball and space. Forcing teams to occupy a smaller space than they would occupy without competition. This is called the Realised Niche.

Two fundamental niches defined by a pair of variables in a two-dimensional niche space. As depicted in Hutchinsons 1957 paper.

Two fundamental niches as depicted in Hutchinson’s 1957 paper.

Catennacio like Niche theory has had plenty of success, for football the Italian club teams of the 60, 70s and 90s and the national team all had success using this tactic, most recently in 2006 where they used an updated version to claim the 2006 world cup final. In ecology, niche theory is alive and well and still remains widely implemented and debated in fields such species distribution modelling and community ecology.

There is a long history in the ecological literature that aims to understand what processes drive interactions between species. One of the most famous in the Lotka-Volterra competition model. The Lotka–Volterra model is a pair of differential equations, used to describe the dynamics of biological systems in which two species interact. If we want explicitly explore this concept, we can think of the famous Lotka-Volterra model in terms of football.

The following equations represent a model of continuous logistic competition between two species. For the purposes of our soccer analogy the equations represent the success of two competing teams.

\Large \frac{d N_{1}}{d t} = r_{1}N_{1}(1-\alpha_{11}N_{1}-\alpha_{12}N_{2})
\Large \frac{d N_{2}}{d t} = r_{2}N_{2}(1-\alpha_{21}N_{2}-\alpha_{21}N_{2})

r_{i} Represents individuals produced per individual per unit time. As we are thinking of this in terms of football, this might be the appearance of new players? However as that doesn’t really make sense in terms of a football match, let’s think of this as number of passes achieved at each time step. A proxy for a teams competitive edge.

\alpha_{ii} Is the effect of intra-specific competition between individuals of species i on the overall population size. Once again in terms of football this might represent the tempo as game is played at? If the tempo (speed a team plays) is fast, you might expect to out play the opposition. If you play really fast then we might expect teams to get tired, their passing starts to fall apart and eventually you lose due to exhaustion. Additionally, if a team’s tempo is too slow and you lose (Unless you’re the Spanish team, as we’ll discover in the example below).

\alpha_{ij} Is the negative effect of inter-specific competition of species j has on the growth rate of species i or vice versa. For the purposes of my football example, we are going to think of this as pressure from the opposing team.

Now it pains me to revisit the 2010 Football World Cup because of my Dutch heritage. But for the sake of science, I’ll use the 2010 WC final as an example of this model applied to football.  And as we know the result (Spain won), I have fixed the match to enable Spain to win. As you can see in the example figure below.

A Lotka-Volterra model that depicts the outcome of the 2010 EWC final.

A Lotka-Volterra model that depicts the outcome of the 2010 EWC final.

 

Now if you want to see what are the effects of \alpha_{ii} and \alpha_{ij}  on the outcomes of our 2010 EWC final you can visit this Shiny app I’ve created to see what happens. What happens if you increase the tempo (\alpha_{ii}) and pressure (\alpha_{ij}) Spain exert on the Dutch team? What happens if the Spanish team drank to much Sangria and play terribly? Have a play and see if you can alter the outcome of 2010 Ecological World Cup.

Well that’s it for another week of Ecological World Cup 14. Next week I might try and explore the links between Total Football and Neutral Theory.

And as Alan Partridge would say: “That was liquid football.”

Posted in Uncategorized | 1 Comment

The Ecology Football World Cup. “Match” One: The Golden Team and IBT.

It’s time. Time for what? Time for the World Cup! And to discuss the links between ecology and football. I’m going to write a weekly analogue between a key football tactic and an established ecological theory. Comparison one, the Golden Team and Island Biogeography Theory (IBT).

So let’s go back to the 1950s. The beginning of a football revolution. A few world cups completed and the inventors of the game England, remain trophy less. As Crick and Watson unlock the structure of the DNA molecule. They were watching their own England play the Golden Team in the “Match of the Century”. England, a team almost never beaten on home soil. Versus the Magical Magyars, an Hungarian national football team of the 1950s. The Mighty Magyars won the famed Match of the Century. Ferenc Puskás, Sándor Kocsis and their coach Gusztáv Sebes changed football for the better. Sebes energised his players to be versatile. Driving his players to shift position across the field. Moving his team away from established tactics of fixed positions and deterministic roles.

Puskas - The Nucleolus of the Golden Team.

Puskas – The Nucleolus of the Golden Team.

In ecology, Island Biogeography Theory was also a game changer. The idea of IBT kicked off in the 1960s by Robert MacArthur and E.O. Wilson. IBT states immigration and extinction influences species richness on undisturbed islands. Distance of islands drives immigration rate.  Islands that are further away from a source population are less likely to receive colonisers. Larger islands are likely to have more species. This is due to large island have more diverse habitats that minimise local extinction.

IBT

Equilibrium Model form MacArthur and Wilson (1967)

If we think of species numbers as goals.  Players as islands. The football pitch as a network of islands. And scoring goals as species immigration and conceding goals as extinction. A player is more likely to score a goal if they receive good passes from their team mates. This is analogous to the IBT idea of an islands getting more species if they are close to a source population. Players in space maintain possession of the ball, and in turn, they are less likely to lose the ball and concede goals. As in IBT, more spacious islands support more species.

So that’s analogy one completed. A simple one to start. But as we can see, early developments in football tactics suggest that space and connectivity are important for goals. The same is true for diversity.

Next match: Catenaccio and Niche Theory. Stay tuned.

Posted in Uncategorized | Tagged , , , , , , | Leave a comment

It’s been a while

In all news Skip’s research. I’ve got a lot of blogging to catch up on. A few post to come on upcoming adventures to US and Canada.

Cool R trick and tips with Knitr, creating R Packages to make your work flow much more efficiently and a pretty handy little code checking tool.

I found this little gem the other day, made me laugh.

seashell

Posted in Uncategorized | 1 Comment

Read and listen.

A Spectrogram of music. The colour intensity is logarithmic!

I’m one of those people that loves to listen to music while working and reading. So here are a few papers I’m enjoying along with some tracks to accompany them.

A classic paper (May, 1976) deserves a classic tune. There is some challenging maths. But I have this (unrealistic but awesome) image of Lord Robert May bopping to Blue Oyster Cult and (Don’t fear) The Reaper and doing some differential equations.

I also like this paper by Olivero et al 2012 (Olivero et al., 2012), this is a good methods paper that provides some interesting insights to defining biogeographic regions. Being a methods paper you sometimes feel like you can hardly keep you head above water, but these tracks should help you swim.

A main part of my PhD is trying to describe drivers of deep-sea biodiversity at oceanic and global scales. This paper looks at the temporal variability in ocean productivity (Behrenfeld et al., 2006). One it’s an interesting paper, but two, temporal variability should be explored when looking at predictors of biodiversity. Here is a classic Thelonious Monk album which is well known for its treacherous tempo changes. If you’re not a Jazz fan, maybe pop is more your thang

And finally,  Belanger et al (2012) predict global biogeographic structure for shallow marine environments. An interesting paper that compares existing qualitative bioregionalisation to models built on correlations between species occurrence data and environmental predictors. Not surprisingly temperature came out on top.

Until next time,

Skip.

Behrenfeld, M.J., O’Malley, R.T., Siegel, D.A., McClain, C.R., Sarmiento, J.L., Feldman, G.C., Milligan, A.J., Falkowski, P.G., Letelier, R.M. & Boss, E.S. (2006) Climate-driven trends in contemporary ocean productivity. Nature, 444, 752-755.

Belanger, C.L., Jablonski, D., Roy, K., Berke, S.K., Krug, A.Z. & Valentine, J.W. (2012) Global environmental predictors of benthic marine biogeographic structure. Proceedings of the National Academy of Sciences, 109, 14046-14051.

May, R.M. (1976) Simple mathematical models with very complicated dynamics. Nature, 261, 459-467.

Olivero, J., Márquez, A.L. & Real, R. (2012) Integrating Fuzzy Logic and Statistics to Improve the Reliable Delimitation of Biogeographic Regions and Transition Zones. Systematic Biology,

Posted in Uncategorized | 1 Comment

Ten “crack that paper” commandments

Biggie

Ten commandments so you can publish during your PhD,an ode to Pia Lentini and The Notorious B.I.G. Over the weekend I was trying to write up my first PhD chapter as a paper, when The Notorious B.I.G got shuffled into my playlist. It’s been 20 years since Biggie released ‘Ready to die’. This got me thinking about a talk a Post Doc in QAECO lab gave on publishing during your PhD. So I decided to procrastinate from the task at hand and write an ode to Pia and Biggie Smalls. I give you the:

Ten “Crack That Paper” Commandments (play this track and recite the commandments with me!)

I been in this game for a year, doing it for me love of animals
There’s rules so you don’t publish shit, I wrote me a manual
A step by step booklet for you to get
Your game on track, not your wig pushed back

Rule nombre uno: never start the show
without, a plan ready to unfold, modify as you go.
The calendar decreed planning specially
If your times backed up, head down bum up

Number two: never not let em (supervisors) know your next move
Don’t you know em bad boys remain in silence
Take it from your highness (uh-huh)
You need to squeeze them chaps, ask them for their tricks and tips.

Number three: do tasks mu-tip-ly
If your draft is receiving feedback, don’t put your feet up
Ramp your analysis up, or even do a quick graph
Start thinkin’ about your next chapter and get a step up

Number four: know you heard this before
Never deny your timeline supply (Be realistic with your timelines ~3 Drafts per paper)

Number five: Ignore superiors comments is a liability
I don’t care if they’re a nonce, snub remarks and your paper will bounce

Number six: that first draft edit, dead it
You think a lab-head writing your manuscript, shit forget it

Seven: this rule is so underrated
Pitch your paper and ideas to a journal list updated
Methods and “Nature” don’t mix like no fingers and an itch
The wrong pitch and you’re in serious shit

Number eight: never keep no hate in you
Them cats that review your paper get knocked back too

Number nine shoulda been number one to me
If you ain’t kickin’ goals and your paper is rejected by the journal police
If reviews think your works decent by ain’t glisten
Just remember chance plays a role, there is no need to be frettin’

Number ten: a strong word called consignment
Strictly for all men, not just the best men
If you ain’t put in the hours they’ll say “hell no
Cause they gonna want that paper as pure as the drive snow

Follow these rules you’ll have mad papers to post up
If so, all the way to the top
Slug out your paper, watch your thesis stand up
Marker read your papers, they’ll say this is too good to pass up

Hopefully Pia will provide an updated link to her of slides. But in the mean time here are some useful links to look at when thinking about publishing your work: Joern Fischer’s writing blog: http://writingajournalarticle.wordpress.com/
and Corey Bradshaw’s: http://conservationbytes.com/2012/10/22/how-to-write-a-scientific-paper/.

Cheers Skip.

And if you haven’t heard the original here it is! (Apologies to Biggie, hopefully he doesn’t roll over in his grave).

Posted in Uncategorized | 1 Comment

Conference Presentation at 13th International Deep-sea Biodiversity Symposium

I’m speaking next Monday the 3rd of December at the 13th International Deep-sea Biodiversity Symposium in Wellington, New Zealand.

I’ll be speaking on beta-diversity in the deep-sea. My talk will focus on briefly describing the bathyal environment around Australia and New Zealand, the methodological approach I’ve used to create maps of beta-diversity based on presence-only records and I will attempt to test the hypothesis does deep-sea biogeographical patterns resemble those seen in shallow marine environments.

If you can’t make it to Wellington and are keen to have a look at a few of my maps and slides they are available for your viewing pleasure.

Posted in Uncategorized | Leave a comment