Showing posts with label programming. Show all posts
Showing posts with label programming. Show all posts

01 March 2012

Date and Time

Disclaimer: this post is about date and time on earth in human time frames, relativity concerns do not apply.

Time information is one the most misunderstood types of data. This misunderstanding usually culminates in the question “Should time be stored as local time or as UTC?”, the answer to which is “Neither, it should be stored as time.”

Points in time and their human representation

A point in time is a concept, a written date is the human readable representation of this concept, the name of the point in time. The point in time is the same everywhere but it may be called by different names.
The Atlantis space shuttle launched for the last time at “11:29 am, July 8 2011 EDT” which just means the time in Florida when the last space shuttle was launched was 11:29 in the morning. At that time it was only 8:29 in Los Angeles, and someone in Sydney would have to get up at 1:29 in the morning of the 9th to catch the liftoff live.
“11:29 am, July 8 2011 EDT” is just a representation for the time the of the lauch, “01:29 am, July 9 2011 Sydney time” and “3:29 pm, July 9 2011 UTC” are other representations for the same point in time.
The problem is that as humans we are used to dealing with non complete time representations. When someone asks us to call them “at four”, we infer, from context, to mean “today at four in the afternoon local time.” Most of our interactions with time leave some information to context. The most common omitted information is the time-zone, and unfortunately this has been brought over to computer languages.

Time in Computer Programming Languages

Most computer programming languages’ standard libraries offer a type for storing date-time information.
The common internal representation used to express time in these types is usually a number of some unit of time since a specific well defined date. Since this date is date zero it is called the epoch for that language/platform.
For example the C/C++ time_t type is defined as the number of seconds since midnight January 1, 1970 (the Unix epoch;) Java’s Date uses the same  epoch but the units are milliseconds and .Net’s DateTime stores the number of 100 ns units (called ticks) since midnight January 1 of year 1 CE (the common era epoch.)
Using units since the epoch makes date and time calculations and comparisons simple, but it is decidedly not  human readable, which is why standard libraries also provide ways for date types to be converted to and from human readable formats and here is where problems usually start.
In C/C++ the epoch is not well defined (i.e. it does not contain timezone information) and timezone information is also absent from the tm structure which is used to parse time_t variables into the components of date and time representations (month, yesr, hour, etc.mponents. It is almost imposible in C/C++ to do time translations without ending up with time_t variables using epochs in different time zones.
Java’s Date class used to look like a wrapper for C/C++ time functions, this was fixed in JDK 1.1, which obsoleted those functions and added the GregorianCalendar class to handle conversions. Java also specifies the epoch’s timezone as UTC, but since Date does not store timezone information this is merely a convention.
.Net did almost the same blunder in version 1 and 1.1, but where a bit more frank about it and added the DateTimeKind enumeration which marks a DateTime as UTC  i.e. number of ticks since UTC epoch, local i.e number ticks since the epoch in the timezone configured into the computer; or undefined i.e. timezone is unknown.
It is not possible to reliably compare between times with different timezone epochs so a variable specifying a utc time can be compared with any other utc time variable, local time variables can only be compared with local time variables whose value is known to have originated in the same timezone. In all other cases only variables with times more than 24 hours apart can be reliably compared.

How should time be stored?

Twenty years ago data did not travel much, so there was not much concern about implications of using local dates and time in a global setting. In today’s interconnected world this is no longer the case.
Now, pun intended, it’s time we revisit the question at the start of this post:

“Should time be stored as local time or as UTC?”

The answer remains the same “Neither, it should be stored as time.”, but to qualify this answer:

There is no UTC time or local time, a point in time is a point in time, UTC and local are just representations. It is possible to use a date/time variable that stores time using a UTC epoch, a local epoch or even an unspecified epoch, but only the first one is a well defined point in time; the second relies on context, namely the timezone configured in the computer; and the third is not properly defined at all.
When using date/time types that do not offer timezone information use UTC time.
The question now is “Is this enough?”
For applications that just need to record when something happens, it is indeed enough.
Timezone only matters when the time needs to be formatted in a human readable representation; at other times, date/time information can live happily as an opaque date/time variable.
There are applications where the local time of the event is important.
When I look at a photo taken on a holiday, I’d like to know that it was taken at two in the afternoon local time.
If the photo in this example contained location information theoretically that would be enough to figure out the local time, but in practice the offset from UTC in time zones changes occasionally and the period when daylight time savings are in effect for a specific timezone changes much more often. This is information that is not readily available, so the best approach is to store the  time using a well defined epoch, and the offset from UTC.
For run-time .Net provides the DateTimeOffset data type which stores a local DateTime and its offset from UTC. Java does not provide a class for storing time and timezone information, but provides the SimpleTimeZone class which can be used to convert Date objects to local string representation (remember in Java all Date objects should be UTC.)
For persistent storage, data types which support timezone offset in databases should be used if available.
If everything else fails time can be persisted as a standard string representation that includes timezone offset information such as the full date and time in ISO 8601, but this approach requires parsing the strings before any computations.

31 March 2010

Error Handling

Error handling is a much debated topic.
A large portion of the discussion centers on the technical aspect of error handling: when to throw exceptions and when to return error status, how to incorporate error reporting facilities, etc. but the subject is much wider.
In this post I’d like to explore the topic at a higher level and focus on the validation code, the code that generates the errors: Where to write validation code, and how much validation code to write so that the errors can actually be handled.

There is no fail safe program
The naive approach to validation is to regard each method (function, procedure, subroutine, etc.) as a separate entity. As such each method needs to, on the one hand validate all inputs, i.e. accept only valid inputs and reject all others with an error, and on the other hand handle all possible errors generated by called methods.
The naive approach tries to achieve fail safety by attempting to cover every eventuality. This is not only unrealistic, but also impossible:
  1. It is not possible to validate all input.
  2. Let’s look at a simple example: binary search. For binary search to work it requires the random access list which is one of its inputs to be sorted. Can we verify the input to binary search is sorted? Not in O(log(n)) time complexity.
  3. It is not possible to handle all errors.
  4. There are whole sets of error types programmers choose to ignore: hardware failures, memory corruption, bugs in driver software, etc. Most programs won’t even try to cope with memory allocation errors (out of memory exceptions.) Some of these failures are handled by the operating system others require special hardware. Unless the program is critical, chances are it is perfectly acceptable for the program to crash under these circumstances.
  5. Validation code is not bug free.
Bugs are statistical beings the more validation code written the higher the chance a bug during validation will cause the program to fail.

Managing failures with contracts
A powerful approach for designing reliable programs is called Program by Contract.
Using this approach each software unit (method/class/library) in the program is designed using a Contract.

The contract contains three parts:
  1. A set of valid entry states usually stated as pre-conditions that must be met before calling the unit, and the set of inputs recognized by the unit.
  2. The action performed by the unit for each valid entry state.
  3. The state of the unit (or post-condition) after the call completes.
The important thing to remember about contracts is that they not only specify how the unit succeeds, but also how it fails.
Let’s take the file IO operations for instance. Attempting to open a file without sufficient permission fails, but fails in an expected way (throwing a security exception or returning an appropriate error code.)
What happens if the function is not called from a valid entry state, i.e. pre-conditions are not met or the input is not recognized?
In this case the result is undefined, as in anything can happen.

To illustrate this let’s look at the binary search function once more. The contract for binary search specifies that given a value and a random access sorted list, the search can find (in O(log(n)) time complexity) the index of the value in the list if it exists or the insertion point for the value so the list remains sorted otherwise.
If the list is not sorted then binary search returns some index in the list as it is not possible to define what index is returned without knowing the exact values of the list.

Using contracts allows us to write less code.
If we can ensure an entry state that causes an error is never used then the error will never occur and we don’t need to write error handling code.
Furthermore if we know an input is validated by a method in the façade, we don’t need to write validation code for the same input in an internal method; and approach called “perimeter defense”.
How does having less validation and error handling code make for more reliable programs? It may sound counter-intuitive but reliability is not obtained through more validation.
Contracts define how a software unit behaves and not having to write unnecessary code leaves time to concentrate on the two activities that do increase reliability:
  1. Designing the program so that preconditions are always met so only expected errors can happen and are handled.
  2. Writing unit tests to verify the software units comply with the contract.
There are other ways in which having less unnecessary code increases reliability. The code is more readable and as mentioned validation and error handling code is not bug free. The worst case scenario is for a program to fail due to a bug in error handling code for an error that doesn't happen.

Validation code revisited
As mentioned contracts allows removal of unneeded validation and error handling code. While we definitely do not want to write error handling code for errors that should never happen, we may still want to leave more validation code than strictly necessary in place.
The first and most important reason is to make the code more reusable. Errors that can never occur in one program may be inevitable in another. Having validation code in place means the contract is less onerous on the side of the caller and we can use the code in more situations.
The second reason is to prevent unacceptable harm from unforeseen errors, such as those caused by bugs. Database foreign key constraints are an example of this type of validation. Programs that use a database do not really need foreign keys. Business logic in the middle tier is usually coded with the relations in mind. In fact inserting and removing data is many times less efficient than it could be because of them.
However we still use them to ensure data integrity remains even if there is a bug in the business logic.
The benefits of extra validation must always be weighted against the impact on performance, reliability and readability of the overall program.