Below is a list of some of my most used features in various web developer tools. The tools can do so much more than I described here. Many of the features are somewhat hidden, so perhaps there’s a few things you might find useful too.

Feel free to share any of your favorite features in a JS and AJAX tool in the comment section below.

Microsoft’s Visual Studio.NET

The tool I use by far the most is Microsoft’s Visual Studio.NET 2005. Its script debugging features far surpasses anything I’ve seen in any other tool. The downside is that this only works for Internet Explorer. What I wouldn’t give for this to work with FireFox too.

The debugging experience is as good as the debugging in a .NET application. You get breakpoints, watches, object inspection and manipulation, step-into/over and so on.

This feature is somewhat hidden, because you need to go through a few hidden steps to enable it (this will change with VS2008).

First you need to enable script debugging in Internet Explorer. Go to internet options, and uncheck “Disable script debugging (Internet Explorer)”.

Then go to “View – Script Debugger – Open” and select a running instance of Visual Studio .NET.

In Visual Studio, go to “Debug – Windows – Script Explorer”. This will bring up the window with the loaded page and scripts. Double-click any of the files in the script explorer to open it and set breakpoints, create watches, inspect objects etc. Open the “Immediate window” to execute any JavaScript you enter. Note that these features only are available while you are at a breakpoint.

A little quirk compared to debugging a .NET application is that you can only hover the mouse on the root object, and not on its sub properties. The workaround is to select the object to get its value, as shown above. The good thing is that this even works on methods, so if you select an object, its method and the ending enclosing parentheses, the tooltip will show you the returned value from that method.

If you are developing a web application in VS.NET and run it in debug more, VS.NET will automatically attach to the browser process if you have enabled script debugging.

FireBug

My favorite tool for debugging JavaScript in FireFox is FireBug. It’s the Swiss army knife of web development in FF. One of the cool features that a colleague of mine brought to my attention was a neat little feature for tracking all events that happen on a selected DOM element. This is great for analyzing which events are available for listen for, see the order that events are fired etc.

To use this feature, choose the HTML tab and browser to the element you want to track. Right-click it and select “Log events”.

Switch to the Console tab and try moving the mouse over the element and you will see a continuous output of all events occurring on the element:

You can also inspect the properties of each event by clicking them:

Nikhil’s Web Development Helper

Nikhil Kotari has created a great little plug-in for Internet Explorer to assist primarily AJAX development. The most used features it the request logger, that logs all HTTP requests and provides you with some serializers for interpreting the responses. Deserializers included by default are Text, Hex, Image, Microsoft AJAX Partial Postbacks and JSON. The JSON and Partial postback deserializers gives you a nice little tree-view for inspecting the AJAX responses.

The newest release also features a neat JavaScript object class browser. It interprets JavaScript objects in memory and shows them in a familiar Visual Studio/C# like object browser.

IE developer toolbar

IE developer toolbar is a useful tool for DOM inspection and manipulation. My favorite feature here is that it enables me to select any element and modify the DOM and CSS properties on the fly. This gives you a quick trial-and-error approach to getting your CSS to do what you want without the need for modifying HTML and refreshing the page.

It also gives you quick access to clearing the cache, generating image reports, save out the loaded and the generated HTML, a measure tool, and a bunch of other useful tools for inspecting the page.

Fiddler Tool

Fiddler tool inspects all http request and responses from all running applications. I use this for optimizing responses by checking that gzip encoding is used and pages that should be cached is cached (inspect cache header and result code):

This tool is VERY powerful and also includes scripting, altering request/responses etc, but that is a whole other story.

Fiddler Tool works with any browser that respects Windows’ proxy settings (for instance Safari 3b doesn’t, but IE and FireFox do).

I have lately seen several blogposts confused about which datum and projection Microsoft’s Live Maps (Virtual Earth) and Google Maps use. As most people already know by now, they render the round earth onto a flat screen using a Mercator projection.

I think the confusion comes from that they actually use two spatial reference systems at the same time:

1. Geographic  Longitude/Latitude coordinatesystem based on the standard WGS84 datum.
2. Mercator projection using a datum based on WGS84, BUT modified to be spheric.

So when is what used?

The Javascript API’s use (1) as input when you want to add points, lines and polygons. That is, they expect you to input any geometry in geographical coordinates, and click events etc. will also return geometry in this spatial reference. This is the coordinate system most javascript developers will use. The API will automatically project it to the spheric mercator projection.

If you want to create image overlays, or roll your own tile server on top of the map, you will need to project your images into a spheric mercator projection. The JavaScript APIs are not able to do this for you.

Here’s a bit of facts about the two projections:

The valid range of (1) is: [-180,-85.05112877980659] to [180, 85.05112877980659].

The valid range of (2) is: [-20037508.3427892, -20037508.3427892] to [20037508.3427892, 20037508.3427892]

Well-known Text for (1):
GEOGCS["GCS_WGS_1984",DATUM["D_WGS_1984",SPHEROID["WGS_1984",6378137,298.257223563]],PRIMEM["Greenwich",0],UNIT["Degree",0.0174532925199433]]

Well-known Text for (2):
PROJCS["Mercator Spheric", GEOGCS["WGS84basedSpheric_GCS", DATUM["WGS84basedSpheric_Datum", SPHEROID["WGS84based_Sphere", 6378137, 0], TOWGS84[0, 0, 0, 0, 0, 0, 0]], PRIMEM["Greenwich", 0, AUTHORITY["EPSG", "8901"]], UNIT["degree", 0.0174532925199433, AUTHORITY["EPSG", "9102"]], AXIS["E", EAST], AXIS["N", NORTH]], PROJECTION["Mercator"], PARAMETER["False_Easting", 0], PARAMETER["False_Northing", 0], PARAMETER["Central_Meridian", 0], PARAMETER["Latitude_of_origin", 0], UNIT["metre", 1, AUTHORITY["EPSG", "9001"]], AXIS["East", EAST], AXIS["North", NORTH]]

Proj.4 definition for (1):
+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs

Proj.4 definition for (2) (see here for an explanation of the weird ’nadgrids’ parameter):
+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +no_defs

So why this weird extent for the latitude? First of all, the poles in the Mercator projection extends towards infinity, so at some point they have to cut them off (and who cares about ice anyway - apart from that its melting). If you look at (2) these latitude/longitude values projected into the spheric mercator results in a perfect square, fitting perfectly with squared image tiles, that are simple to sub-divide over and over again, as you zoom in. I expect the reason for the spheric datum is for simplicity and perfomance when reprojecting points from longitude/latitude to screen coordinates. Charlie Savage also has a more mathematical approach to deriving these values.

For a quick introduction to projections, coordinate systems and datums see here.

If you want to know more about how these mapping api's work, keep an eye on Jayant's blog.

Update: We now have an offical EPSG code for the projection. See details here.

Inspired by xkcd, here's my take on it as a javascript developer:

My Profanity Usage by Browser

To be fair to the numbers above, I first of all base my JavaScript on the Microsoft AJAX client library, so usually this works pretty well with IE7. I start out with making stuff work in IE7 and then adjust it to work in FireFox, IE6, Safari etc. This is probably why I spend more time fixing quirks in FireFox than in IE7. I'm currently doing some testing against Safari 3.0beta, and some of the issues I'm having were often resolved by making some subtle (but illogical) changes in the JavaScript, but these changes were often something I've stumbled upon by sheer luck (hence the profanities). It might just as well be because this is a beta browser. Surprisingly I do find myself spending more time working around issues and bugs in FireFox than in IE, but in the end I'm usually able to come up with a script that works in all major browsers and only have very few tweaks that tests against which browser I'm using (thank God for JS base API's that does most of that work).

IE6 wins as the browser that required the most extra code needed, just because of one thing: PNG Transparency. I need objects that need to know whether its using images that are PNG or not, so I can apply the transparency workaround (but at least this is a well-known workaround). It just complicates the JS API, requiring developers to deal with more properties to ensure it works in IE6. I can't wait till the day where we can sign of that browser version.

So why do I primarily develop in IE7, and not FireFox which seems to be the case for a lot of developers? Many reasons:

• The debug integration with Visual Studio.NET is lightyears ahead of FireBug and all the other JS debug tools in FireFox (and is the JS console really all I get in Safari 3.0 on Windows?).
• There are some great tools to browse and modify the DOM, see requests etc. My favorites are "Web Development Helper" which even have JSON and Microsoft AJAX Partial Postback deserializers to help you easier analyze the server responses, and IE Developer Toolbar for viewing the DOM and modifying it on the fly (priceless when you are tweaking your CSS styles).
• It's the most used browser out there, so this is in my mind also the most important one. I'm not saying the others aren't important, but this should ensure that the most used browser also gets the most preliminary testing.
• I know all the non-ECMA-Script properties and methods that IE adds to the DOM, so I also know which properties and methods to avoid. I'm not that familiar with Mozilla's extensions and don't want to risk using methods that doesn't exist in other browsers (that just adds to the number of profanities :-).
• This one is probably going to give me some flaming, but I do think IE7 is the best overall browser.

I've had a lot of requests for the .NET 2.0 based raytracer that I used in my Master Thesis to generate true orthophotos based on complex 3D surface models (not necessarily 2½D). To save me from having to dig it up all the time, I've now posted it here on my blog so you can download it from here.

Basically it will read in a text file with triangles (each triangle defined per line as comma-separated double values X1,Y1,Z1,X2,Y2,Z2,X3,Y3,Z3), create a spatial index using axis-aligned boundingboxes, and perform ray/triangle intersections, returning any triangle intersecting the ray, including distance to triangle and angle of attack.

To save memory on larger DSM, all internal computation is done using floating-point insted of using double precision. This poses a problem for large values like UTM based coordinate systems, where rounding errors will occur when using floating point. The raytracer is internally applying an offset to the center of the data extents to accommodate for this, but it's worth remembering if your DSM covers a vast area. In most cases it's not a big issue though.

The raytracer scales very well (O(log n)), and can handle hundreds of thousands of faces and still raytrace in a few milliseconds (see chapter 7.3.3 of the thesis for details). Performing the required indexing first might be a little slow, so if you want to optimize this, be my guest ;-).

Some uses this engine has been involved in is generating rough worldfiles for aerial images based on absolute camera orientation and a surface model or Simulating Navstar and Galileo GPS availability in dense urban areas. The latter was actually some very cool stuff developed by some students at DTU. You could click anywhere on a map, and it would instantly simulate the visibility of Navstar and Galileo GPS satellites and give you the PDOP values throughout the day, based on a detailed 3D city model. This was used to determine where the zones of GPS based roadpricing should be (you didn't want the zone edges to be close to a place where you had poor accuracy causing people to be charged incorrectly). The raytracer was used for doing the line-of-sight analysis between a virtual car and all the satellites.

Download: Raytracer.zip (86.29 KB)

I finally had a chance to play with Silverlight today. The managed code capabilities is very interesting, so I set out to create a simple performance tester.

I created two basic managed methods in Silverlight. One that multiplies a number and returns a result, and another that does the same thing a specified number of times:

[Scriptable]
public double Multiply(double a, double b)
{
   return a * b;
}
[Scriptable]
public double MultiplyLoop(double a, double b, int count)
{
   double result = 0;
   for (int i = 0; i < count; i++)
      result = a * b;
   return result;
}

This gives me 3 test-cases to compare:

• Multiply two numbers n times in pure javascript.
• Multiply two numbers n times, by calling Multiply(a,b) from JavaScript into managed code n times
• Multiply two numbers n times, by calling MultiplyLoop(a,b,n) from Javascript into managed code one time.

What I found was:

• JavaScript is fairly slow to do this (345 ms for 10 millions calculations).
• Calling a managed method millions of times is VERY slow (IE asked me several times whether I wanted to abort it before it finished)
• Calling a managed method once and do the looping in managed code is VERY fast (0-2 ms).

The conclusion here would be that you shouldn't have methods in managed code that you call a lot from javascript unless they are process-intensive. There seems to be a big overhead in calling into managed code, but as soon as you are in the managed process, its blazing fast.

You can download my test sample here: SilverlightProject1.zip (18.35 KB)  (requires Orcas beta 1 and Silverlight add-on)

Now on to move all my existing .NET libraries into a Silverlight app... :-)

People are often mixing the above as if they were one and the same, so here’s a recap of them. One of the things you often find people saying is that “my data is in the WGS84 coordinate system”. This doesn’t really make sense, but I will get back to this later.

This is a very confusing subject, and I might have gotten a few things wrong myself, so please add a comment and I’ll update it ASAP.

Coordinate systems

A coordinate system is simply put a way of describing a spatial property relative to a center. There is more than one way of doing this:

• The Geocentric coordinate system is based on a normal (X,Y,Z) coordinate system with the origin at the center of Earth. This is the system that GPS uses internally for doing it calculations, but since this is very unpractical to work with as a human being ( due to the lack of well-known concepts of east, north, up, down) it is rarely displayed to the user but converted to another coordinate system.
  
• The Spherical or Geographic coordinate system is probably the most well-known. It is based on angles relative to a prime meridian and Equator usually as Longitude and Latitude. Heights are usually given relative to either the mean sea level or the datum (I’ll get back to the datum later).
  
• The Cartesian coordinate system is defined as a “flat” coordinate system placed on the surface of Earth.  In some projections it’s not flat in the sense that it follows the earth’s curvature in one direction and has a known scale-error in the other direction relative to the distance of the origin. The most well-known coordinate system is the Universal Transverse Mercator (UTM), but surveyors define their own little local flat coordinate systems all the time. It is very easy to work with, fairly accurate over small distances making measurements such as length, angle and area very straightforward. Cartesian coordinate systems are strongly connected to projections that I will cover later.

Sidenote: The geocentric coordinate system is strictly speaking a cartesian coordinate system too, but this is the general terms I've seen used the most when talking about world coordinate systems.

Datums and ellipsoids

Some of the common properties of the above coordinate systems are that they are all relative to the center of Earth and except the Geocentric coordinate system, uses a height system relative to the surface of the earth.

This poses two immediate problems:

• Where is the center of the earth
• What is the shape of the earth?

By now most people should know that that the earth isn’t flat (although there are still some who doubts it). If we define the surface of Earth as being at the mean sea level (often referred to as the Geoid), we don’t get a spheroid or even an ellipsoid. Because of gravitational changes often caused by large masses such as mountain ranges etc, Earth is actually very irregular with variations of +/- 100 meters. Since this is not very practical to work with as a model of earth, we usually use an ellipsoid for approximation. The ellipsoid is defined by its semi-major axis, and either the flattening of the ellipoid or the semi-minor axis.

The center and orientation of the ellipsoid is what we call the datum. So the datum defines an ellipsoid and through the use of a set of points on the ground that we relate to points on the ellipsoid, we define the center of the Earth.  This poses another problem, because continental drift moves the points used to define the points around all the time. This is why the name of a datum usually have a year in it, often referring to the position of those points January 1st of that year (although that may vary).

There are a vast amount of datums, some used for measurements all over the world, and other local datums defined so they fit very well with a local area. Some common ones are: World Geodetic Datum 1984 (WGS84), European Datum 1950 (ED50) and North American Datum 1983 (NAD83).

The most well-known is WGS84 used by the GPS systems today. It is a good approximation of the entire world and with fix-points defined almost all over the world. When it was defined they forgot to include points in Europe though, so the Europeans now have their own ETRS89, which is usually referred to as the “realization of WGS84 in Europe”. The problem here was solely because of continental drift, so they defined some points relative to WGS84 in 1989, and keeps track of the changes. In most use-cases it is of no real importance and you can use one or the other.

I mentioned earlier that people often refer to having their data in WGS84, and you see now why this doesn’t make sense. All you know from that is that the data is defined using the WGS84 datum, but you don’t know which coordinate system it uses.

Read more on Datums and Spheroids.

Projections

The earth isn’t flat, and there is no simple way of putting it down on a flat paper map (or these days onto a computer screen), so people have come up with all sorts of ingenious solutions each with their pros and cons. Some preserves area, so all objects have a relative size to each other, others preserve angles (conformal) like the Mercator projection, some try to find a good intermediate mix with only little distortion on several parameters etc. Common to them all is that they transform the world onto a flat Cartesian coordinate system, and which one to choose depends on what you are trying to show.

A common statement that I hear in GIS is the following “My map doesn’t have a projection”, but this is simply not possible (unless you have a good old rotating globe). Often people are referring to data that is in longitude/latitude and displayed on a map without having specified any projection. What happens is that the system applies the simplest projection it can: Mapping Longitude directly to X and Latitude to Y. This results in an equirectangular projection, also called the “Plate Carree” projection. It results in very heavy distortion making areas look squashed close to the poles. You can almost say that the “opposite” of the Plate Carree is the Mercator projection which stretches areas close to the poles in the opposite direction, making them look very big. Mercator is the type of projection you see used on Live maps and Google maps, but as many often mistakenly thinks, they do NOT use WGS84 for the projected map, although WGS84 is used when you directly input longitude/latitude values using their API (read more on this here).

More on projected coordinate systems

Spatial reference

The spatial reference is a combination of all the above. It defines an ellipsoid, a datum using that ellipsoid, and either a geocentric, geographic or projection coordinate system. The projection also always has a geographic coordinate system associated with it. The European Petroleum Survey Group (EPSG) has a huge set of predefined spatial references, each given a unique ID. These ID’s are used throughout the industry and you can download an Access database with all them from their website, as well as some very good documents on projection (or see the Spatial References website).

So when you hear someone saying they have their data in WGS84, you can often assume that they have longitude/latitude data in WGS84 projected using Plate Carree.  The spatial reference ID of this is EPSG:4326.

Spatial References are often defined in a Well-known format defining all these parameters. The Spatial Reference EPSG:4326 can therefore also be written as:

GEOGCS["GCS_WGS_1984",DATUM["D_WGS_1984",SPHEROID["WGS_1984",6378137,298.257223563]],PRIMEM["Greenwich",0],UNIT["Degree",0.0174532925199433]]

As mentioned Live/Google maps use a Mercator projection, but although their datum is based on WGS84, they use a sphere instead of an ellipsoid. This means that they use the same center and orientation as WGS84, but without applying any flattening. The spatial reference string for their projection therefore becomes:

PROJCS["Mercator Spheric", GEOGCS["WGS84based_GCS", DATUM["WGS84based_Datum", SPHEROID["WGS84based_Sphere", 6378137, 0], TOWGS84[0, 0, 0, 0, 0, 0, 0]], PRIMEM["Greenwich", 0, AUTHORITY["EPSG", "8901"]], UNIT["degree", 0.0174532925199433, AUTHORITY["EPSG", "9102"]], AXIS["E", EAST], AXIS["N", NORTH]], PROJECTION["Mercator"], PARAMETER["False_Easting", 0], PARAMETER["False_Northing", 0], PARAMETER["Central_Meridian", 0], PARAMETER["Latitude_of_origin", 0], UNIT["metre", 1, AUTHORITY["EPSG", "9001"]], AXIS["East", EAST], AXIS["North", NORTH]]

The best session I attended at the Mix was probably Jeff Prosise and Matt Gibbs' session Go Deep With ASP.NET AJAX which was really well-prepared and had a lot of good pointers on what to be aware of when using Microsoft AJAX. It's a must-see if you are using Microsoft AJAX.

Link to session

Earlier today (some would say way too early after a long night at Pure) I attended the session "How to make AJAX Applications scream on the client". This turned out to be more about general patterns in Javascript and not so much about AJAX.

Anyway here are my notes from that session:

Avoid Eval(). Instead use parameterized code.

'Switch' is costly for large sets and grows linearly. Instead use a hashtable and wrap in try/catch.

Getters/setters are inefficient. Use direct access to the variables in an instance.

DOM Instantiating and traversing is slow because it has to go from the JS layer to the DOM layer through COM. Use it as little as possible. Example:

   Bad: for(var i=0;i<100;i++) { document.getElementById('myDiv').innerHTML = i; }
   Better: var div = document.getElementById('myDiv'); for(var i=0;i<100;i++) { div.innerHTML = i; }

Use speculative download. Try and anticipate what the user needs to download next. For instance, on the login page start loading icons that the users will need after logging in. The browser might as well spend the time he is spending on entering his password on downloading stuff. When he then logs in the browser will retrieve the images from the cache instead of having to download them from the server, making the page load much quicker. Try going to the login page of an Outlook Webaccess website and see what happens behind the scenes through the Fiddler tool.

Enable GZIP encoding on the webserver. Most browsers supports it and will result in smaller downloads. Again Fiddler is great for experimenting with this.

You can see the full session online here.

On my way back from the Mix conference in Vegas I was trying to get one of my WPF/E applications to run on the new Silverlight 1.1alpha. It wasn't as easy as I thought it would be because a lot seems to have changed. I finally figured it out after reading Bryant Likes's blog. He already figured it out and shared the solution (Thanks Bryant!):

• Upgrading from WPF/E to Silverlight
• Silverlight Hello World in C# from VS 2005 (How to compile and run .NET based Silverlight apps in VS.NET 2005 instead of Orcas)

If you want to go the "real" route of working with Silverlight 1.1alpha, here's what you need:

• Silverlight 1.1 alpha
• Visual Studio "Orcas" Beta 1
• Silverlight Tools Alpha for Visual Studio "Orcas" Beta 1
• Expression Blend 2 May Preview (Optional) - For designing your XAML
• ASP.NET Futures (Optional) - For ASP.NET AJAX integration and ASP.NET Silverlight Servercontrols (the Media control is a good example on how a cool templatable control can be done)

I saw two really good sessions on Silverlight and .NET this morning. They are online now, and I really recommend that you watch them:

• Building Silverlight Applications Using .NET (Part 1 of 2)
• Building Silverlight Applications Using .NET (Part 2 of 2)

One of the really neat things are that Silverlight also supports LINQ in the client browser. Part 2 shows a cool example on that and how easy it makes it to work with data. Furthermore you can easily communicate between Browser DOM, Javascript and .NET assembly code which makes for some interesting scenarios. Since .NET excution is 400-1000 times faster than JavaScript, you could have your heavy algorithms inside silverlight, and call them from Javascript. If you don't need the presentation-layer, you could just hide it and have it as a library you utilize from JS.

So what is the main differences between v1.0 and v1.1? v1.1 contains:

• Everything from v1.0
• A subset of the .NET 3.5 framework and some extra browser and AJAX specific classes
• XAML extensibility
• Control class (for user controls)
• Sample controls

Currently Silverlight works with Safari, Firefox and IE on both Mac and Windows and soon to be Opera as well. I asked them whether they would also support other platforms like Linux. Basically they didn't have any 'issues' with making that, but were currently not considering it until there is a larger demand for it.

Update: Mono has already announced that they will make sure we get Silverlight support on Linux. COOL!

Today Microsoft demoed their AjaxView application in one of the Mix07 sessions. It acts as a proxy and monitors requests, as well as execution time for each javascript function that gets executed. This is really neat for tracking what methods are putting the most load on the client browser. Now all we need is some built-in tracking of memory leaks ;-)

A tech preview will be available within the next 1-2 months.

Update: You can now see the full session where this was demoed online here.