• Processing (Info)

    • MPC Status Page: Archive (2010 January-December)

    This page describes enhancements to or problems that have occurred with the MPC and scripts and the fixes that have been made.

    Recent problems are listed elsewhere. Index of other older problems..

    Older Enhancements and Resolved Problems

    • Power and Network Outages Dec. 11 and 13
      2010 Dec. 10: The power to CDP will be disconnected from 07:00 EST to 15:00 EST on Saturday, December 11, in order to connect a new electrical service to the Park. In addition, CDP will suffer a network outage from 18:00 EST to 21:00 EST on Monday, December 13, in order to relocate the fiber that provides Internet access into CDP. Neither of these events will (should!) affact core MPC operations, but beta testers of the new web server will be unable to access the new setup during these outages.

    • Last night's DOU MPEC
      2010 Dec. 10: 13:00. Last night's DOU MPEC has been abandoned as a result of a protection issue in copying some important update files over to the Linux systems.

    • Incorrect H values
      2010 Dec. 8: 23:50. Following a report from a user, a problem with the H values for 143 orbits of numbered minor planets published in the November MPCs has been identified and been fixed. The H values on the affected orbits have been fixed in our master files and in the files used by the MPES; the values used by the MPChecker programs will be fixed by the next DOU MPEC. The corrected orbits will also be republished on the next DOU MPEC.

    • November MPCs
      2010 Nov. 20: 14:20. The preparation of the November batch of Minor Planet Circulars will be delayed by a week due to recent events.

    • MPChecker
      2010 Nov. 14: 22:05. In order to get the newly-hosted MPChecker script working, we need to introduce a change in an internal data file that will break the existing MPChecker script. As soon as the new format data files are on-line, we will fix the existing script.
      • 23:05. In a testament to the OS design, the existing MPChecker script works with (what we thought would be an imcompatible change in the format of) the new-format data files used by that service.

    • Erroneous designation messages
      2010 Nov. 9: 12:40. Over the past day or so, some automated process has been appending unnecessary designation lines to the files that are used by AUTODES to send designations back to observers. We are trying to locate which automated process is misbehaving.

    • PHAs not being flagged in MPES
      2010 Nov. 9: 12:00. Following a query from a user of the MPES, it was determined that the MPES was not indicating that recent discoveries were PHAs. The problem was traced to a change in an internal procedure. This problem is believed to be fixed.

    • Change in time-keeping
      2010 Nov. 7: 18:00. Following an overnight problem with the automated changeover of system clocks from EDT to EST, the decision was made to set all system clocks to UTC. The problem caused five machines to remain set to EDT and this caused the despatch of the many automated batch jobs to occur only once per hour, rather than once every minute or two. All automated batch jobs are now running on their normal schedule.

    • No MPC Batch in October
      2010 Oct. 19: 17:50. There will be no MPC batch in October. The next mid-month MPS batch will be issued around October 31.

    • Apparent Unavailability of Scully
      2010 Oct. 12: 11:35. Over the past few days we have received a number of reports of the apparent unavailability of Scully. In every case, a check of the server logs shows that Scully was serving documents throughout the reported problem period. It seems that every report was made at a time of very heavy server load. We are currently setting up a new (much more powerful) web server machine that we hope to have on-line shortly.
      • 20:10. A check of yesterday's server log shows that 16642 calls to the cgi scripts from 4580 different IP addresses.

    • Complex-wide Power Outage Saturday, September 18
      2010 Sept. 14: The power to most of the buildings at 60 Garden Street will be shut off from 07:00 to 19:00 EDT on Saturday, September 18, in order to perform a power systems upgrade. The Computation Facility will begin shutting down its systems at 05:00. While the power outage does not affect any of the MPC's computer systems (since they are in locations that will be unaffected by the power outage), the loss of power elsewhere will cut all network traffic to/from CfA. We will not be able to send/receive e-mail during the outage: any messages sent to us should be queued automatically by the sending system and resent once network connectivity is restored. All CF webservices will be unavailable. Access to MPC cgi scripts will not possible due to the network outage. A similar power outage is planned for Saturday, October 2.

    • MPECs 2010-R13 to -R20 and -R22
      2010 Sept. 3: 19:05. It seems that the issuance of the MPECs 2010-R13 to -R20 and -R22 was affected by a disk error. The circulars are being remailed. We apologise if you get two copies of these circulars.

    • Reuse of circular number "MPEC 2010-Q01"
      2010 August 30: 09:30. The MPEC circular number 2010-Q01 was inadvertently reused by the overnight DOU MPEC routine. It should have been 2010-Q41. The problem was caused by a reorganization of disk storage and an apparent failure to copy the latest version of the MPEC index file. Last night's DOU MPEC has been renamed 2010-Q41 in the on-line version.

    • Network interruption August 12
      2010 August 4: 12:23. We have received notice that Harvard will be performing maintenance on a core router on Thursday, August 12, 05:00-07:00 EDT. There may be intermittent outages of connectivity in this period.

    • HTML-4.01 compliance
      2010 July 11: 20:00. We have received a query about our use of the HTML-4.01 compliance icons on most of our pages. If you test the compliance you will see five errors reported. These errors are not due to anything on our pages but are the result of a single line of usage tracking code that is automatically appended to each file returned by the server. The invalid nature of the line has been reported to the hosting company, but they appear to be unwilling to fix it. In the near future, the main MPC site will move back to computers hosted at CfA. Some time thereafter, the mirror site will be moved to a new, permanent location.

    • Preprepared Ephemerides
      2010 July 4: 21:45. The bulk of the preprepared ephemerides that used to be available at /iau/Ephemerides have been restored. A new feature is the ability to download KML files of the paths of the comets for display in Google Earth.

    • Last night's DOU MPEC
      2010 May 31: 12:24. Last night's DOU MPEC has been abandoned as a result of a system failure overnight.

    • Web Server Unavailability
      2010 April 12: 14:40. We are disabling the MPC webserver for an hour or so to help us track down a network issue.
      • 15:40. The MPC webserver has been restarted.

    • Possible Network Disruption (March 10)
      2010 March 9: 15:45. On Wednesday, March 10, starting at 09:00, we will be reconfiguring our VMS cluster. For outside users the most visible sign of this reconfiguration will be (hopefully) short outage of the MPC webserver. Since the reconfiguration involves changes in our DNS registration, it may take time for the changes to propogate outside CfA.
      • March 10: 14:00. The network reconfigurations have been completed. The changes are visible internally.
      • 17:10. As of now, the DNS changes are not visible from my home system.
      • 22:40. It is probable that the inability of outside use to access cgi services of the MPC webserver is due to blocking of http traffic at the sub-net level. This suspicion is apparently confirmed by the fact that my home systems see the new IP address for the MPC webserver This problem has been reported to the Computation Facility.
      • March 11: 10:10. The cgi scripts are again visible externally.

    • Official MPC Mirror Site II and Mid-Month MPS
      2010 Feb. 18: 12:20. Efforts to get the MPC mirror site fully up and functional are proceeding slowly. Progress will remain slow until such time as access is restored to the disabled user accounts in the CF.

      It is worth pointing out that it seems unlikely the CF-hosted MPC website will be updated in the future. The reasons for this are not relevant for public discussion, but we will agree if this is the decision of the CF. All future updates will therefore occur only on the www.minorplanetcenter.org site. Simply replace "www.cfa.harvard.edu" or "cfa-www.harvard.edu" with "www.minorplanetcenter.org" and you should find the desired resouce on the new site (eventually!).

      We can make no guarantees at this stage as to when the next mid-month MPS batch will be released. We will be looking at the preparation procedure in the next few days.

    • Official MPC Mirror Site
      2010 Feb. 15: As a result of the hacker intrusion mentioned in Feb. 12 note, we are bringing the official MPC mirror site on-line immediately. Please understand that this early release is solely to make certain data available to certain professional collaborators. The completeness of the mirror site is currently minimal and the current unavailability of access to the main site means that it is not trivial to copy the necessary data to make it complete. In addition, most of the routines that copy data to the main website are not yet modified to copy data to the mirror site.

    • Successful hacker intrusion at CfA
      2010 Feb. 12: 11:00. We have been informed by our Computation Facility that the CF account of a MPC staff member has been hacked. Consequently, that account has been disabled, as has the network access to the MPC's Linux cluster. At this point, we have no evidence that the core VMS cluster was compromised.

      The upshot of this hack is that updating of the MPC website is not possible and circulars cannot be mailed out via their normal route. We have set up alternate routes for mailing circulars, but recipients who check the From: line of messages before allowing delivery of the message will probably not receive the circulars. There is nothing we can do at this end about this problem. It is also unclear when we will be able to update the webpages.

      Static pages such as the MPES main page remain functional, but periodically-updated pages such as the NEOCP do not. Observers are advised to use the mirror page links.

    • SMTP E-mail Interruption (January 29)
      2010 Jan. 26: 16:57. We have been informed that the CF's SMTP server will be off-line for approximately 30 minutes beginning at 06:05 EST on Friday, January 29, to allow OS patches to be installed. Attempts to send e-mail to us during this period will fail, but the mail should queue up on the sending computer for a subsequent resend attempt.

    • Web Services
      2010 Jan. 2: 18:30. The web services are functioning, but the webserver is getting pounded by MPChecker request. We will examine the logs and, if the requests are coming from the same user, will block access from their IP address.

    • Internal Network Problems II?
      2010 Jan. 1: 14:00. There appears to be a problem with network connectivity. It is unclear at this point whether there is any relation to the Dec. 30 problems. The problem is causing the web services to hang. We are investigating.
      • Jan. 2: 11:15. Attempts to resolve the problem remotely were unsuccessfully (although we believe we know where the problem originated). We have just arrived in the office and will be attempting manual fixes shortly.
      • Jan. 2: 12:00. We believe we have fixed the problem and are checking the cluster status and the state of the numerous automated processes that handle the bulk of the MPCs work. Normalcy should be restored shortly.