Bug: Flags lost when updating "Printer friendly" log page

When generating a ``printer friendly'' monthly log display, values in the ``Flag'' column were not encoded as {\tt hidden} items in the result page. If the user then did an update from this page, all checked flags would be lost. I added code to embed {\tt hidden} items for checked flags, which will propagate them back on an update. Note that due to the way we integrate the rung and flag fields on updates there is no need to embed their values as hidden fields. It's only due to the odd design of HTML/CGI which doesn't allow you distinguish the absence of an editable field from a check box not checked that we require this work-around. (Reported by user ``rhittom''.)

Bug: Set UTF-8 mode for user names in persistent logins

The {\tt cookie::storeCookie} and {\tt cookie:testCookiePresent} methods failed to set UTF-8 mode when reading and writing the cookie token file for persistent logins. This caused warning messages when sorting cookie user names in the administrator Persistent Login Manager page. This change will force all users with non-ASCII login names to log back in, as their cookies will not match those stored with the incorrect file encoding. As it happens, there were only two such and both were inactive accounts, so I just purged the persistent logins for them myself.

Bug: Eliminate unnecessary sorting in enumerating accounts, sessions, and persistent logins

When enumerating public accounts for the browse public accounts page, or all user accounts, open sessions, or persistent login tokens for the administrator, we unnecessarily sorted the names of the files in the respective directories before retrieving them into the hash used to build the displayed table. Since the table generation code sorts the hash keys, there is no need to sort the file names, which can be very time consuming when these directories get large. If these directories get very much larger, it may make sense to read them serially and perform the {\tt grep()} on them within the loop rather than bringing the directory into memory and using the {\tt grep()} function as presently done.

Feature: Choose Active/Inactive/All accounts on public and administrator browse

To expedite display of public accounts, I added a drop-down box to the ``Browse public user accounts'' item on the Utilities page which allows the user to select active, inactive, or all accounts with active the default. An active account is defined as one with a transaction within the last 30 days. In addition, the user can switch between the display of active, inactive, or all public accounts on the Browse Public Accounts page.

Under Administrator Functions on the Utilities page, the administrator may choose, when managing user accounts, to display active, inactive, or all user accounts (as for public accounts, active means a transaction within the last 30 days), and may switch selections from the Account Manager page. In addition, the administrator can access a user account directly by name to view, purge logs, or delete. For the latter two functions, the administrator password must be entered as a confirmation before the button is pressed.

Bug: Error message for administrator accessing nonexistent account

If the administrator attempted to access (view) a nonexistent account, the application would exit with an error and yield a blank screen. With the original account manager, this could happen only if the user deleted the account between the time the list was displayed and when the administrator attempted access, but with the direct access facility, the error would occur whenever the administrator entered an invalid account. I added an explicit error message for this circumstance which includes the invalid account name.

Bug: Error messages for administrator purge or delete account

The error messages generated when the administrator attempts to purge the logs or delete a nonexistent account name were missing a space before the name of the account---fixed.

Feature: Direct access to public accounts by name

With more than 2500 public accounts, it takes almost forever for the list of public accounts to load. This is very irritating to people who regularly check on their friends' progress. What I'd like to do is add the ability, when viewing a public account, to check a box to make it (or remove it from being) a ``friend''. In the Utilities page, a drop-down list of friends will be displayed, from which you can select a public account to access with a single click. To address the immediate problem, until I manage to put all of this machinery in place, I've added a simple text box and ``View'' button below the ``Browse public user accounts'' item in the Utilities menu. The user can simply enter the name of the public account in the text field and press the button to go directly to the account. This isn't as convenient as a list of friends, but it's a lot better than waiting for the monster form to load. The direct name access form is also included in the Utilities page displayed when viewing a public account, allowing direct transfer from one public account to another.

Bug: Divide by zero in sparse monthly log trend analysis

If only one weight was present in the most recent log and no entries existed in the previous month's log for the preceding week, the {\tt trendfit::fitSlope()} method would divide by zero because there were insufficient points to fit a linear trend. I added code to the method which reports a zero trend slope when insufficient points are available to fit a slope. Note that while this was discovered testing the new method of computing the trend for the current month, the bug was present prior to the change.

Feature: Trend analysis for current monthly log

When displaying the current (or most recent) monthly log, the trend analysis displayed beneath the chart was based on the data plotted in the chart. At the start of a month, when there were only a few data points, this could result in day-to-day instability in the trend analysis. I added code to test whether the most recent log is being displayed and, if so, a trend for the last seven days is computed using a {\tt history} object, even if that requires retrieving days from the previous month. This also ensures that the trend reported on the monthly log page will always be identical to that shown for the last week in the Trend Analysis page.

Bug: Race condition in ClusterSync mkdir and rmdir transactions

Due to race conditions (for example, processing cluster synchronisation transactions while a global server synchronisation is underway), it is possible for an {\tt mkdir} or {\tt rmdir} transaction to fail because the directory in question already exists or has already been removed. To avoid a possible loop retrying such transactions, I added tests for these cases which deem the transaction successful if its intended effect has already taken place.

Bug: Month option for Web badge

Added a ``Month'' option for Web badge generation. This is something intended from the start, but omitted due to scatterbrained developer. (Reported by Kees Huyser.)

Bug: Unicode CGI POST arguments in Perl 5.8.8

A change in the handling of the \verb+decode_utf8+ function between Perl 5.8.5 and 5.8.8 broke decoding of CGI arguments containing ISO-8859 and Unicode characters when received as POST arguments. Whereas before we needed to read POST arguments with ``{\tt :utf8}'', now it appears we need to use ``{\tt :raw}'' unconditionally. How much would you like to bet we'll be changing this back somewhere down the road.

Admin: Production server switch

Updated configuration to accommodate the switch from Server1 to Server0 as the primary production server. We now install test versions on Server1 and production versions on Server0, and permit non-{\tt https} logins to Server1 without the security warning.

Bug: Imported exercise rung fields containing spaces

It was possible by using quoted fields to import a native mode CSV file whose exercise rung field included embedded spaces. This would cause a native database file to be written which, when {\tt monthlog::load} attempted to load it, would cause the parser to abort. I added code to {\tt monthlog::importCSV} to delete any spaces in an exercise rung field.

To cope with existing database entries with spaces in exercise rung fields, I added code to {\tt monthlog::load} to delete all spaces from the record. An entire database can be cleaned up by performing a trend recalculation. (Reported by Tom Gunter.)

Bug: Cycling of cluster synchronisation log files

When the cluster synchronisation log file was cycled after receipt of a {\tt HUP} signal, the cycling was erroneously performed both in the signal processing code and in the main loop code which responds to the signal. I removed the file cycling from the signal handler, where it is vulnerable to race conditions within Perl and the underlying C library.

Feature: Line buffering of cluster synchronisation log file

Added code to set automatic buffer flush for the ClusterSync log file when it is initially opened and cycled after receipt of the {\tt HUP} signal. This allows those who monitor the log file with, for example, {\tt tail~-f} to see the complete log item for a transaction without waiting for the buffer to be flushed.

Modified the cluster synchronisation log file generation to skip the ``Results:'' line if the command produced no output.

Feature: Recovery from transient cluster synchronisation failures

Implemented recovery from transient and permanent cluster synchronisation transaction failures. Previously, if any error occurred reading, verifying, or executing a cluster synchronisation transaction, the {\tt ClusterSync.pl} program would crash, suspending cluster synchronisation until it was restarted. Unfortunately, there were a number of circumstances in which such errors could occur, the most common being cases where a race condition between queueing the transaction and {\tt ClusterSync}'s processing of it caused an incomplete file to be read (transient), and those where a crash of the process queueing the transaction caused an incomplete file to be written to the transaction directory (persistent).

When a cluster sync transaction fails, for whatever reason, it is placed into a failed transaction hash whose key is the transaction file name and whose value is an array containing the number of times the transaction has been tried and the next time the transaction should be retried. On subsequent passes through the transaction directory, failed transactions are skipped unless their retry time has arrived, whereupon they are retried and, if they fail, their try count is incremented and the next attempt count updated.

If the transaction eventually succeeds, it is closed out normally and removed from the failed transaction hash. If the transaction fails again, its try count is increment and if it has reached the limit, the transaction is deleted from the transaction directory and the failed transaction hash. Failre to delete the transaction from the transaction directory remains fatal to the {\tt ClusterSync} program.

The intervals between retries of a failed transaction and the number of failures which cause a transaction to be abandoned are set by configuration parameters.

Feature: Paper log form generation

Completed the implementation of a facility for generating and printing paper log forms for people who wish to log offline and then transcribe the data to the application later. A new ``Print paper log forms'' item on the Utilities menu displays a form which allows the user to select the first and last month and year (the form is preset to the default of all months in the current year). The current, previous, and next years may be selected. (If the user sets the end date before the start date, they are silently swapped.) When the ``Generate'' button is pressed, a new window opens with the log document in it, and after a one second delay to allow the page to render, a print command is queued (these features require JavaScript to function; if it is absent, the log document opens in the same window as the request form and the user must print it manually and return to the application with the ``Back'' button). A paged media style sheet is used to insert page breaks so that each monthly log prints on its own page.

Bug: Caption in historical charts with multiple days per pixel changes with chart size

Historical chart generation with multiple days per pixel could report different values for the trend analysis and flag fraction in the caption depending upon the chart size (and hence the number of days aggregated into each horizontal pixel). This was because {\tt getDays} is not guaranteed to examine every day in the interval, particularly in the case of long intervals and small charts. I added code to {\tt drawChart} in {\tt history.pm} to call {\tt analyseTrend} for the entire interval to perform the analysis and use the values it computes for the caption, instead of those computed on the fly by {\tt getDays} which may depend upon the chart scale. (Reported by Jim Hollcraft.)

Bug: Excel format CSV import with space before exercise rung

If an Excel-format CSV record contained a space before a single-digit exercise rung field, the record would be skipped as not parsable. I modified the test pattern to allow leading (but not trailing) spaces. Records of this type are created by the Palm HDread program when a day has an exercise rung between 1 and 9 and the -o option is used to generate Excel-format CSV. (Reported by Ömer Ay.)