• Processing (Info)

    • MPC Status Page: Archive (2005 July-December)

    This page describes enhancements to or problems that have occurred with the MPC and scripts and the fixes that have been made.

    Recent problems are listed elsewhere. Index of other older problems..

    Older Enhancements and Resolved Problems

    • Lists of Dates of Last Observation
      2005 December 31: 00:10. The updating of these lists is off-line while a switch over to using Earth MOIDs is made. Normal service should be restored later today.
      • December 31: 17:53. Work is ongoing. Some of the conversions are rather fiddly...
      • 18:04. It also seems that the conversion affected some files used by various scripts, e.g., the MPChecker. Hopefully, tonight's DOU MPEC will fix any such problems.
      • January 1: 18:00. All known problems with the changeover have been attended to.

    • Full user disk
      2005 December 24: 11:45. A user's home disk filled up overnight. Not sure why, as there was over 5 GB free on the device as of last night. A load of automated procedures failed. They are being restarted. Last night's DOU MPEC also did not complete--the bits that failed are being rerun.

    • Processing Stall and Repeated ACKs
      2005 December 14: 10:15. A stall in the automated processing queue yesterday has been cleared. One consequence of clearing this stalled queue is that some observers may have received multiple ACKs for their messsages. This was unavoidable.

    • Power Outage Planned for Dec. 7
      2005 November 30: 15:15. We have been informed that our building will be without power from 17:00 to (no later than) 19:00 EST on Wednesday, December 7. This outage is necessary in order to make necessary repairs to electrical switching gear that provides power to most of the buildings. Since the actual down time is not known in advance, we will be shutting down the MPC computer system starting at 16:30. Most of the Web scripts will be unavailable during this outage and e-mail will not get delivered to our machines. E-mail (sent to @cfa addresses) will be stacked up on the CF mailserver and will get delivered when power is restored.
      • December 7: 14:31. We will now begin shutting down our systems starting at 16:00, in light of information received about when the CF is shutting down their systems. Some core CF functions will be on backup power during the outage, so restoration of the MPC website and web services after power is restored should be rapid.
      • December 7: 18:45. Power was restored a short while ago and our cluster is back on-line.

    • Network Connection Failure to CF Webserver
      2005 November 9: 23:30. It appears that a network service server on one of the CF boxes failed around 20:40 (for the second time in two days). Until this service is restarted, we will not be able to update any pages on our website. As a direct consequence, tonight's DOU MPEC is being abandoned.
      • November 10: 12:55. We have been informed that, following a third failure of the network interface in the CF machine, the motherboard in the affected machine was replaced. Tonight's DOU MPEC should be issued as normal.

    • Network Interruptions November 2
      2005 October 27: 17:14. We have just been informed that the CF will be upgrading some network components between 17:30 and 20:30 EST on November 2. While we are told that these upgrades will not affect the connectivity of our cluster, it will impact communications to and from CfA. E-mail, if it is delayed, will be stored on the CF mailhub and delivered when possible. Web connections, particularly access to scripts, may be flaky during this period.

    • Truncated Designations in Unusual Object Lists
      2005 October 25: 21:35. It is been noticed, following a comment by a user, that the provisional designations in the lists of unusual objects are truncated. The links from those pages to MPES ephemerides are correct. A fix should be in place shortly.
      • 21:40. A fix has been made and the affected internal files are being regenerated. Externally-visible files will be updated on the next update cycle.

    • New Orbits in MPES
      2005 October 24: 08:35. Because of complications in the preparation of the October batch of MPCs there has been a long period (lasting a couple of days) where new numberings of minor planets are visible on our website, but elements are not available for these objects through the MPES. It is expected that this anomaly will be corrected by this evening.

    • DOU MPEC
      2005 October 24: 08:30. Last night's DOU MPEC has been abandoned. Any files that get updated as a result of DOU issuance will not be updated. Normal schedule should be resumed tonight.

    • E-mail outage Saturday October 22
      2005 October 20: 17:49. We have been informed by the Computation Facility that the CF mail node will be shutdown at 08:00 EDT on Saturday, October 22 in order to bring up new hardware. It is expected to be down for about two hours. During this time, e-mail to cfa.harvard.edu addresses will bounce.

    • Residual blocks for numbered minor planets
      2005 October 20: 17:30. The directory blocks for the numbered minor planets are being moved into a new directory structure. The shifting is being done as a very low priority job. There will be periods where individual residual blocks are unavailable.

    • Power Glitch at Observatory
      2005 October 8: 10:20. It seems that there was a power glitch which affected the Observatory buildings at around 05:30 this morning. Core IT operations in the CF remained up and the MPC/CBAT Computer Network was also unaffected.

    • MPES names and principal designations of numbered objects
      2005 September 30: 11:00. A user reported an incorrect principal provisional designation was displayed for a numbered object. An investigation showed a misalignment in the files of numbers/designations used by the MPES. A slight recode of one routine has been made to prevent this happening in the future.

    • NEAObs data files
      2005 September 25: 11:00. While investigating a report of an "odd" report on desirability of further observation in the MPES, it was noticed that a number of required data files were not being updated as scheduled. The problem was tracked down to the incomplete removal of an object now classified as a comet. The nightly routine that updates these files has been modified to make it more robust to this kind of problem and the affected data files are being updated. Normal service should be resumed shortly.

    • Possible Service Interruptions in August II
      2005 August 22: 16:55. The VMS V8.2 upgrades from the current V7.3-2 are continuing. The webserver was upgraded this morning. The final three machines will be upgraded tomorrow and our new machine will be brought on-line. There may be brief periods when the webserver is unavailable tomorrow.
      • August 23: 14:00. All current machines have been upgraded. Normal service should have resumed. The new machine can't be brought up yet as the CF has not yet completed the "new machine" tasks it is responsible for. Its installation into the cluster should not affect the other machines. No timetable for this new installation is available.

    • Power outage Friday night through Saturday II
      2005 August 19: 04:00. We were alerted 45 mins ago that the power had been restored to the Observatory. We have rebooted our machines but the networking components down in the CF have not yet been powered back on. Only our four machines that are connected via a mini-hub have booted into the cluster.
      • 05:00. Waiting for CF to arrive... Playing games.
      • 06:00. Waiting for CF to arrive... Playing games.
      • 07:00. The networking components have been powered on. Other services of the CF may take longer to come back.
      It is worth remarking that one of our old VAXes has been up for 1622 days (just under 4.5 years).

    • Possible Service Interruptions in August
      2005 August 19: 10:10. The VMS V8.2 upgrades from the current V7.3-2 have been proceeding smoothly. There will be no web service when the web server machine is upgraded and there may be brief periods of unavailability while other machines are being upgraded. The OS upgrades will continue on Monday.

    • Power outage Friday night through Saturday I
      2005 August 18: 12:56. We have just been informed that there is an urgent electrical problem at the Observatory that needs fixing asap. The power to the entire complex will be shut off at 17:00 EDT Friday 19th. The work is expected to take 12 to 24 hours (must be one mother of a problem...) and all MPC/CBAT/ICQ and CfA webservices will be off-line during this period. We intend to start taking our machines down sometime around 16:30. It is unclear at this point if alternate-power arrangements will be made for the CF's mail node.
      • 13:34. We now have some more information. It seems the main electrical service into the Observatory is "seriously defective" and must be repaired "promptly". The word "explode" was used in conversation by someone in the know...
      • 14:03. The Saturday morning DOU MPEC will be cancelled, as will this weekend's mid-month batch of MPS.
      • 18:22. It looks as if the CF's mail node will not be powered up during the outage, so e-mail to cfa addresses will not be delivered and will not be queued by any system at this end. Depending on your e-mail software, your system may attempt to redeliver the message at intervals for several days after the initial attempt.
      • August 19: 11:43. We now know that the CF will start shutting down its machines starting at 15:00 EDT. The CF mail node may go down well before 17:00.

    • MPCORB mirror site
      2005 August 5: 13:22. A user has reported that the MPCORB mirror site has not been updated since Aug. 2. We are investigating.
      • 16:24. Attempting to run the mirroring script manually results in a "connection refused"-style error message. We are contacting the sysman at the mirror site.
      • August 10: 16:21. The sysman was on vacation. He has informed us that the sshd daemon on his machine stopped on Aug. 2. He has restarted it.
      • August 11: 12:34. The nightly update did not work. Attempting to run the script manually produces an odd error message that we have reported to the Computation Facility.
      • 13:05. The problems seem to be fixed. The mirror script is being run manually. Tonight's automatic update is expected to work. The non-standard epoch files for NEAs should get updated at that time.

    • Truncated New Observation Format documents
      2005 August 5: 11:00. While attempting to make a small alteration to the New Observation Format document, it was realised that the .PS and .pdf versions are truncated at the end of the twelfth page. Attempts to render the entire document in these formats have not yet been successful, for reasons that are not clear. The other-format versions of the primary-version Techwriter file are complete.
      • August 8: 11:09. We have been informed by the author of Techwriter that our problem is related to an occasional problem with PS output (it is apparently fixed in the next release of the program). In the meantime, he has informed us of a workaround. New versions will be up shortly.

    • Possible Network Interruptions on August 3
      2005 August 2: 11:16. We have been informed by the Computation Facility that there will be a brief interruption in the Observatory's Internet connection sometime between 07:45 and 08:00 EDT on August 3. The interruption should not last for more than a few minutes. The reasons for this interruption have not been given.

    • Possible Service Interruptions in August
      2005 July 27: 21:40. We are planning on upgrading the OS on all our cluster machines to V8.2 from the current V7.3-2. We are intending to undertake this work starting on or about Aug. 19. During the upgrade process, on-line services may be unavailable for brief periods.

    • MPES
      2005 July 21: 12:00. A user reported a stack dump when requesting ephemerides for some TNOs. A problem with the code not expecting to see a certain required file with zero length was located and fixed. Another user's report of seeing some debugging output in the returned ephemerides (under certain specific circumstances) has also been corrected.

    • Service Disruption July 12
      2005 July 12: 11:00. There will be disruption to the MPC web services starting at 14:00 EDT on Tuesday, July 12. During this period many MPC services will be unavailable. It is hoped that the disruption will not last more than 30 minutes. This disruption is necessary to do some physical relocating of computers.
      • July 12: 14:45. Again starting somewhat later than intended, all systems are now functioning normally.

    • Service Disruption July 11
      2005 July 7: 14:22. There will be disruption to the MPC web services starting at 11:00 EDT on Monday, July 11. During this period many MPC services will be unavailable. It is hoped that the disruption will not last more than 30 minutes. This disruption is necessary to do some physical relocating of computers.
      • July 11: 12:19. We started somewhat later than planned, but the machines are now all relocated and we are checking that everything is running as normal.

    • MPC Search facility
      2005 July 5: 15:40. The Computation Facility has reported that our the search facility now has the necessary index files for it to function as intended.

    • MPCs in ADS
      2005 July 5: 15:35. The missing batches of MPC author information have been submitted to the ADS. When they update their indexes (probably within a few days), the author information for all batches of MPCs issued on or after 2003 June 14 should be available on-line. Older issues will be added as time (and inclination) permits...

    • DOU MPEC
      2005 July 1: 08:30. Last night's DOU MPEC has been abandoned. Any files that get updated as a result of DOU issuance will not be updated. Normal schedule should be resumed tonight.