Password/Username Retrieval Error

On Feb 3 Rebecca J reported that the username and password retrieval function is not working.

From the page http://i2u2-dev.crc.nd.edu/elab/cosmic/login/retrieve-username-password.jsp, attempting to use either entrybox generates one of the two errors

Error: unable to send message. Error: unable to send message. javax.mail.MessagingException: Could not connect to SMTP host: smtp.mcs.anl.gov, port: 25; nested exception is: java.net.NoRouteToHostException: No route to host
Error: unable to send message. Error: unable to send message. javax.mail.MessagingException: Could not connect to SMTP host: smtp.mcs.anl.gov, port: 25; nested exception is: java.net.ConnectException: Connection timed out

Obviously something is attempting to access ANL mail servers (smtp.mcs.anl.gov) despite the fact that they no longer serve the !@i2u2.org domain.

The page is generated by the repository file common/src/jsp/login/retrieve-username-password.jsp, which originally contained three references to elabs@i2u2, an alias that redirected to Edit. These references, however, were in the form of user-readable output Strings and were not used to determine which address was used for retrieval.

On 23Feb2016,

  dvt4:4.0-ND-dev jgriffith$ grep -r --exclude-dir=.svn 'elabs@i2u2.org' ./*

gave hits only for the three instances in common/src/jsp/login/retrieve-username-password.jsp.

This file is deployed to the four files on VM i2u2-dev

  ./tomcat/webapps/elab/(cms-tb | cms | cosmic | ligo)/login/retrieve-username-password.jsp

It's also found in the two in situ VM i2u2-dev files

  ./local-settings/common/resources/classes/elab.properties
  ./tomcat/webapps/elab/WEB-INF/classes/elab.properties

Besides these,

 jgriffi8@i2u2-dev:/home/quarkcat/sw$ sudo grep -rF --exclude="*\.svn*" --exclude-dir=i2u2svn -D skip 'elabs@i2u2.org' ./*

gave no other hits.

I attempted to commit changes to branch 4.0-ND-dev on 23Feb2016, but I was stopped by an SVN error (I had originally checked out 4.0-ND-dev and 4.0-ND-prod in a manner that prevented me from subsequently committing). The SVN error was fixed on 3Mar2016. On Mon 7Mar2016 I made corresponding changes to common/src/jsp/login/retrieve-username-password.jsp of branch 4.0-ND-dev. I committed the change in branch 4.0-ND-prod to the repository, but evidently I neglected to commit branch 4.0-ND-dev.

On 17Mar2016, I noted that deployment in the intervening time (almost certainly Sat 12Mar or Sun 13Mar) had not fixed the problem.

On the VMs, grep -r for "@i2u2" gave

i2u2-prod

 ./local-settings/common/resources/classes/elab.properties (elabs@i2u2)
 ./tomcat/webapps/elab/WEB-INF/classes/elab.properties (elabs@i2u2)
 ./www-php/boinc/html/user/.svn/text-base/HelpDeskRequest.php.svn-base (2) (help@i2u2)
 ./www-php/i2u2-library/LocalSettings.php (help@i2u2)
 ./www-php/quarknet-fellows-library/LocalSettings.php.default (2) (help@i2u2)
 ./www-php/quarknet-fellows-library/LocalSettings.php (help@i2u2)

i2u2-dev

 ./local-settings/common/resources/classes/elab.properties (elabs@i2u2)
 ./tomcat/webapps/elab/WEB-INF/classes/elab.properties (elabs@i2u2)
 ./tomcat/webapps/elab/cms-tb/login/retrieve-username-password.jsp (3) (elabs@i2u2)
 ./tomcat/webapps/elab/cms/login/retrieve-username-password.jsp (3) (elabs@i2u2)
 ./tomcat/webapps/elab/cosmic/login/retrieve-username-password.jsp (3) (elabs@i2u2)
 ./tomcat/webapps/elab/embedded/login/retrieve-username-password.jsp (3) (elabs@i2u2)
 ./tomcat/webapps/elab/ligo/login/retrieve-username-password.jsp (3) (elabs@i2u2)

in a cleaned-up notation where (#) indicates the number of occurrences in a file and (C) represents that the occurrence is within a comment. None of these occur in comments despite the fact that I left original lines commented when I changed common/src/jsp/login/retrieve-username-password.jsp, so those should have "elabs@i2u2" in comments.

Double-checking the repository branches,

dvt4:4.0-ND-dev jgriffith$ grep -r "@i2u2.org" --exclude-dir="i2u2svn" ./*
 ./boinc/html/user/HelpDeskRequest.php (C)
 ./common/src/java/gov/fnal/elab/analysis/notifiers/DefaultAnalysisNotifier.java (C)
 ./common/src/java/gov/fnal/elab/analysis/notifiers/UploadNotifier.java (C)
 ./common/src/java/gov/fnal/elab/util/ElabMemory.java (C)
 ./common/src/java/gov/fnal/elab/util/ElabUtil.java (C)
 ./common/src/jsp/login/retrieve-username-password.jsp (C)(4)
 ./config/old/usr/local/apache/conf/local.d/00-Server.conf (C)
 ./config/www17/etc/apache2/sites-available/i2u2 (C)
 ./config/www17/etc/apache2/sites-available/i2u2-ssl (C)
 ./config/www18/etc/apache2/sites-available/i2u2 (C)
 ./config/www18/etc/apache2/sites-available/i2u2-ssl (C)
 ./config/www18/home/quarkcat/sw/www-php/i2u2-library/LocalSettings.php (C)
 ./config/www18/home/quarkcat/sw/www-php/quarknet-fellows-library/LocalSettings.php (C)
 ./library/LocalSettings.php (C)
 ./testing/jmeter/cosmic/monitor-data-timing.jmx

The last is a reference to "jmeter-cosmic@i2u2", which I intend to fix but is not related to the current problem. Thus, there are no relevant occurrences of "@i2u2.org" in branch 4.0-ND-dev as of 17Mar2016. Similar searches on branch 4.0-ND-prod returned substantially similar results.

It almost looks as if branch 4.0-ND-dev was never deployed to VM i2u2-dev. In fact, my local changes were never even committed to the repository. To fix this, I committed changes in common/src/jsp/login/retrieve-username-password.jsp to branch 4.0-ND-dev and deployed branch 4.0-ND-dev to VM i2u2-dev.

After deployment, the five retrieve-username-password.jsp files on VM i2u2-dev did indeed update with changes I made on 23Feb2016. This left

  jgriffi8@i2u2-dev:/home/quarkcat/sw$ sudo  grep -r "@i2u2.org" --exclude-dir="i2u2svn" ./*
  ./local-settings/common/resources/classes/elab.properties:retrieveResetEmail=elabs@i2u2.org
  ./tomcat/webapps/elab/WEB-INF/classes/elab.properties:retrieveResetEmail=elabs@i2u2.org
  <...>

the two in situ files to be changed. At least, I think they're in situ files. The local branch 4.0-ND-dev does have

  dvt4:4.0-ND-dev jgriffith$ find . -name "elab.properties"
  ./common/resources/classes/elab.properties
  ./config/www13/home/quarkcat/sw/local-settings/common/resources/classes/elab.properties
  ./config/www17/home/quarkcat/sw/local-settings/common/resources/classes/elab.properties
  ./config/www18/home/quarkcat/sw/local-settings/common/resources/classes/elab.properties

but these appear to be irrelevant (the first has no email addresses at all, while the other three are for dead machines). I manually changed both files on the VM i2u2-dev, one at a time and restarting Apache after each update. The username retrieval function continued to give the same error, though.

On Fri 18Mar2016, I discovered that grep -r does not follow symlinks, of which the local filesystems have several, including quarkcat/sw/tomcat/, which is a symlink to wherever Tomcat is installed on the VM or localhost. Thus, it can miss files that are located within symlinked directories. Curiously, it doesn't always do this: one can note above that grep -r on i2u2-dev and i2u2-prod did return files within the symlinked tomcat/ directory. I have no idea why.

Anyway, grep -R does follow symlinks, and I should have been using it all along. Re-checking i2u2-dev with this returns

  jgriffi8@i2u2-dev:/home/quarkcat$ sudo grep -R "@i2u2.org" --exclude-dir="i2u2svn" ./*
  [sudo] password for jgriffi8: 
  ./sw/tomcat/work/Catalina/localhost/elab/org/apache/jsp/cms/login/retrieve_002dusername_002dpassword_jsp.java (S)(3)
  ./sw/tomcat/work/Catalina/localhost/elab/org/apache/jsp/cosmic/login/retrieve_002dusername_002dpassword_jsp.java (C)(4)
  ./sw/tomcat/work/Catalina/localhost/elab/org/apache/jsp/ligo/login/retrieve_002dusername_002dpassword_jsp.java (S)(3)
  ./sw/tomcat/webapps/elab/WEB-INF/classes/elab.properties (C)
  ./sw/tomcat/webapps/elab/cms-tb/login/retrieve-username-password.jsp: (C)(4)
  ./sw/tomcat/webapps/elab/cms/login/retrieve-username-password.jsp (C)(4)
  ./sw/tomcat/webapps/elab/cosmic/login/retrieve-username-password.jsp (C)(4)
  ./sw/tomcat/webapps/elab/embedded/login/retrieve-username-password.jsp (C)(4)
  ./sw/tomcat/webapps/elab/ligo/login/retrieve-username-password.jsp (C)(4)
  ./sw/local-settings/common/resources/classes/elab.properties (C)

Everything looks as it should. There are NO non-comment references to "@i2u2" anywhere within the quarkcat/ directory of VM i2u2-dev. Yet, the error messages indicates something is sending to smtp.mcs.anl.gov. Back to square one.

Instead of searching for settings, it may be more useful to trace the sending of emails within the code itself. In branch 4.0-ND-dev, common/src/jsp/login/retrieve-username-password.jsp contains the line

       String result = elab.getUserManagementProvider().sendEmail(to, subject, emailBody);

elab is an object of type Elab (common/src/java/gov/fnal/elab/Elab.java).

elab.getUserManagementProvider() is an object of type ElabUserManagementProvider (common/src/java/gov/fnal/elab/usermanagement/ElabUserManagementProvider.java).

The class DatabaseManagementProvider (common/src/java/gov/fnal/elab/usermanagement/impl/DatabaseUserManagementProvider.java) implements ElabUserManagementProvider.

DatabaseManagementProvider includes the method sendEmail(), used in retrieve-username-password.jsp. This method is appears to be a straightforward implementation of JavaMail.

sendEmail() includes the properties block

        properties.put("mail.smtp.host", "smtp.mcs.anl.gov");
        properties.put("mail.smtp.port", "25");
        properties.put("mail.smtp.auth", "true");
        properties.put("mail.smtp.starttls.enable", "true");

which is clearly where attempts to send mail via smtp.mcs.anl.gov originate. It's not immediately clear how to alter these to fix the password retrieval problem. Comparison to the Help Desk code is useful, since we know it works.

The Help Desk Request page /boinc/html/user/HelpDeskRequest.php sends mail through the function send_report_via_email(), included from /boinc/html/inc/email.inc. This function returns a value of mail(), which is a standard PHP function for sending emails. It is configured through php.ini, the main PHP configuration file, in the [mail function] block.

There are several php.ini files:

  jgriffi8@i2u2-dev:~$ locate php.ini
  /etc/php5/apache2/php.ini
  /etc/php5/cli/php.ini
  /home/quarkcat/sw/i2u2svn/config/old/usr/local/etc/php.ini
  /usr/share/php5/php.ini-development
  /usr/share/php5/php.ini-production
  /usr/share/php5/php.ini-production.cli

Some of these are due to the fact that Apache and CLI often use separate php.ini files. The most reliable way to determine which is the relevant config file is to place a new file (I called it test.php) into the same directory as the Help Desk code. This is the directory /home/quarkcat/sw/www-php/boinc/html/user/, which shows up as http://i2u2-dev.crc.nd.edu/elab/ligo/teacher/forum/test.php on the website.

The file contains only the line

  <?php phpinfo(); ?>

and opens the info page when accessed through the browser. On i2u2-dev, this page contains

Configuration File (php.ini) Path          /etc/php5/apache2
Loaded Configuration File                  /etc/php5/apache2/php.ini
Scan this dir for additional .ini files    /etc/php5/apache2/conf.d

which indicates that /etc/php5/apache2/php.ini is the relevant php.ini file for anything involving HelpDeskRequest.php. This file has the [mail function] block

[mail function]
  ; For Win32 only.
  ; http://php.net/smtp
  SMTP = localhost
  ; http://php.net/smtp-port
  smtp_port = 25

  ; For Win32 only.
  ; http://php.net/sendmail-from
  ;sendmail_from = me@example.com

  ; For Unix only.  You may supply arguments as well (default: "sendmail -t -i").
  ; http://php.net/sendmail-path
  ;sendmail_path =

  ; Force the addition of the specified parameters to be passed as extra parameters
  ; to the sendmail binary. These parameters will always replace the value of
  ; the 5th parameter to mail(), even in safe mode.
  ;mail.force_extra_parameters =

  ; Add X-PHP-Originating-Script: that will include uid of the script followed by the filename
  mail.add_x_header = On

  ; The path to a log file that will log all mail() calls. Log entries include
  ; the full path of the script, line number, To address and headers.
  ;mail.log =
  ; Log mail to syslog (Event Log on NT, not valid in Windows 95).
  ;mail.log = syslog

Which is almost entirely comment, but shows that localhost and port 25 are the SMTP settings used to send Help Desk emails.

(As a side note, all php.ini files on the system have localhhost and port 25 specified. Also, there are no SMTP settings in any of the additional configuration files loaded through the directory /etc/php5/apache2/conf.d/)

This leaves only the JavaMail authentication settings in sendEmail() to take care of. These are not specified explicitly in php.ini, but the PHP mail() function does not handle authentication natively. Instead, you'd need to use Sendmail or another mail server to handle this. By inference, then, if Help Desk emails work without authentication, then Password Reset emails ought to as well.

I changed the sendEmail() method of the class common/src/java/gov/fnal/elab/usermanagement/impl/DatabaseUserManagementProvider.java from

      properties.put("mail.smtp.host", "smtp.mcs.anl.gov");
      properties.put("mail.smtp.port", "25");
      properties.put("mail.smtp.auth", "true");
      properties.put("mail.smtp.starttls.enable", "true");

to

      properties.put("mail.smtp.host", "localhost");
      properties.put("mail.smtp.port", "25");
      properties.put("mail.smtp.auth", "false");
      properties.put("mail.smtp.starttls.enable", "false");

and it works!

Disabling authentication is a little worrying at first, but I think it is harmless in this case. Since the sending of emails is hard-coded, outside users cannot use JavaMail as a server, authenticated or not, without access to the code on the server. ANL probably used authentication because it was sending through the (or one of the) primary mail servers smtp.mcs.anl.gov, which requires greater security. Here, we're sending directly from localhost.

Status: FIXED 28Mar2016

-- %USERSIG{WIKINAME - 2016-03-23}%

Comments