Thursday, May 18, 2017

Rules for user-facing text

1. Do not use 'Please' in user-facing text
2. Do not use contractions in user-facing text. e.g. Change "won't be" to "is not". 

Monday, May 15, 2017

Modify port number in .xml configuration file

1234

[root@sandbox jetty]# cat a.sh
#!/bin/bash
port=$1

sed -i 's/[0-9]\+<\/Set>/'"${port}"'<\/Set>/g' jetty.xml


Example:
[root@sandbox jetty]# ./a.sh 5678

Friday, May 12, 2017

How to make an Ambari service support https only

I have already set up a new Ambari Service MyDemo which provides a quick link supporting http only. Here is the quicklink.json for it:


{
 "name": "default",
 "description": "default quick links configuration",
 "configuration": {
  "protocol": {
   "type": "http_only",
  },

  "links": [
   {
    "name": "mydemo_ui",
    "label": "MyDemo UI",
    "requires_user_name": "false",
    "component_name": "MYDEMO_SERVER",
    "url": "%@://%@:%@",
    "port":{
     "http_property": "mydemo.http.port",
     "http_default_port": "18042",
     "regex": "^(\\d+)$",
     "site":"mydemo-site"
    }
   }
  ]
 }
}

The quick link now points to http://sandbox.hortonworks.com:18042

Next, there is new request to make MyDemo support https only. I think the change should be quite straightforward - just replace all "http" in the configuration file with "https". But, a big BUT here, it does NOT work as expected. the quick link points to:
    http://sandbox.hortonworks.com
The protocol is incorrect and the port number is missing.

Based on the documentation from Ambari:
    https://cwiki.apache.org/confluence/display/AMBARI/Quick+Links
The protocol type can be "https_only" if the 'checks' are empty. But it just dose not work.

After several trials, here is the working version:


{
 "name": "default",
 "description": "default quick links configuration",
 "configuration": {
  "protocol": {
   "type": "https",
   "checks":[
    {
     "site":"mydemo-site"
    }
   ]
  },

  "links": [
   {
    "name": "mydemo_ui",    "label": "MYDEMO UI",
    "requires_user_name": "false",
    "component_name": "MYDEMO_SERVER",
    "url": "%@://%@:%@",
    "port":{
     "https_property": "mydemo.https.port",
     "https_default_port": "18043",
     "regex": "^(\\d+)$",
     "site":"mydemo-site"
    }
   }
  ]
 }
}

Note that the major change is to add the "checks" with only 'site' which is always true.
Now the quick link is pointing to https://sandbox.hortonworks.com:18043

No bug is NOT fixed eventually!


BTW:  Service log directory: /var/lib/ambari-agent/data

Monday, May 8, 2017

Configure embedded Jetty server with one .xml configuration file

String jettyConfig = "/tmp/jetty/keystore/jetty.xml";
XmlConfiguration configuration = new XmlConfiguration(new FileInputStream(jettyConfig)) ;
configuration.configure(server);

Embedded Jetty server to support https - Server Parameters in .XML configuration files

1. Java Code to start up Jetty Server:
        // load jetty xml files in order: jetty.xml -> jetty-ssl.xml -> jetty-ssl-context.xml -> jetty-https.xml
        List<String> configurations = new ArrayList<String>();
        configurations.add("jetty.xml");
        configurations.add("jetty-ssl.xml");
        configurations.add("jetty-ssl-context.xml");
        configurations.add("jetty-https.xml");


        String jettyConfigDir = "/myconfig/jetty/";
        XmlConfiguration last = null;
        List<Object> configuredObjects  = new ArrayList<Object>();
        for (String configFile : configurations) {
            InputStream configStream = null;

            File xmlConfiguration = new File(jettyConfigDir + configFile);
            if (xmlConfiguration.exists()) {
                configStream = new FileInputStream(xmlConfiguration);
            } else {

            }

            XmlConfiguration configuration = new XmlConfiguration(configStream);
            if (last != null) {
                configuration.getIdMap().putAll(last.getIdMap());
            }
            configuredObjects.add(configuration.configure());
            last = configuration;
        }

        // first object is a Server instance because of the jetty.xml
        Server server = (Server) configuredObjects.get(0);

2. jetty.xml
<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "http://www.eclipse.org/jetty/configure_9_3.dtd">

<!-- =============================================================== -->
<!-- Documentation of this file format can be found at:              -->
<!-- http://wiki.eclipse.org/Jetty/Reference/jetty.xml_syntax        -->
<!-- =============================================================== -->

<!-- =============================================================== -->
<!-- Configure a Jetty Server instance with an ID "Server"           -->
<!-- Other configuration files may also configure the "Server"       -->
<!-- ID, in which case they are adding configuration to the same     -->
<!-- instance.  If other configuration have a different ID, they     -->
<!-- will create and configure another instance of Jetty.            -->
<!-- Consult the javadoc of o.e.j.server.Server for all              -->
<!-- configuration that may be set here.                             -->
<!-- =============================================================== -->
<Configure id="Server" class="org.eclipse.jetty.server.Server">
 <!-- =========================================================== -->
    <!-- extra server options                                        -->
    <!-- =========================================================== -->
    <Set name="stopAtShutdown"><Property name="jetty.server.stopAtShutdown" default="true"/></Set>
    <Set name="stopTimeout"><Property name="jetty.server.stopTimeout" default="5000"/></Set>
    <Set name="dumpAfterStart"><Property name="jetty.server.dumpAfterStart" deprecated="jetty.dump.start" default="false"/></Set>
    <Set name="dumpBeforeStop"><Property name="jetty.server.dumpBeforeStop" deprecated="jetty.dump.stop" default="false"/></Set>

    <New id="httpConfig" class="org.eclipse.jetty.server.HttpConfiguration">
      <Set name="secureScheme">https</Set>
      <Set name="securePort"><Property name="jetty.ssl.port" default="1234" /></Set>
      <Set name="outputBufferSize">32768</Set>
    </New>

</Configure>
3. jetty-ssl.xml
<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "http://www.eclipse.org/jetty/configure_9_3.dtd">

<!-- ============================================================= -->
<!-- Base SSL configuration                                        -->
<!-- This configuration needs to be used together with 1 or more   -->
<!-- of jetty-https.xml or jetty-http2.xml                         -->
<!-- ============================================================= -->
<Configure id="Server" class="org.eclipse.jetty.server.Server">
  <New id="sslHttpConfig" class="org.eclipse.jetty.server.HttpConfiguration">
    <Arg><Ref refid="httpConfig"/></Arg>
    <Call name="addCustomizer">
      <Arg><New class="org.eclipse.jetty.server.SecureRequestCustomizer"/></Arg>
    </Call>
  </New>

  <!-- =========================================================== -->
  <!-- Add a SSL Connector with no protocol factories              -->
  <!-- =========================================================== -->
  <Call  name="addConnector">
    <Arg>
      <New id="sslConnector" class="org.eclipse.jetty.server.ServerConnector">
        <Arg name="server"><Ref refid="Server" /></Arg>
        <Arg name="factories">
          <Array type="org.eclipse.jetty.server.ConnectionFactory">
          </Array>
        </Arg>
        <Set name="host"><Property name="jetty.ssl.host" deprecated="jetty.host" /></Set>
        <Set name="port"><Property name="jetty.ssl.port" deprecated="ssl.port" default="1234" /></Set>
        <Set name="idleTimeout"><Property name="jetty.ssl.idleTimeout" deprecated="ssl.timeout" default="30000"/></Set>
      </New>
    </Arg>
  </Call>
</Configure>
4. jetty-ssl-context.xml
<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "http://www.eclipse.org/jetty/configure_9_3.dtd">

<!-- ============================================================= -->
<!-- SSL ContextFactory configuration                              -->
<!-- ============================================================= -->

<!--
  To configure Includes / Excludes for Cipher Suites or Protocols see tweak-ssl.xml example at
     https://www.eclipse.org/jetty/documentation/current/configuring-ssl.html#configuring-sslcontextfactory-cipherSuites
-->

<Configure id="sslContextFactory" class="org.eclipse.jetty.util.ssl.SslContextFactory">
 <Set name="KeyStorePath">
   <Property name="jetty.base" default="." />/<Property name="jetty.sslContext.keyStorePath" deprecated="jetty.keystore" default="../config/\
jetty/jetty.keystore"/></Set>
  <Set name="KeyStorePassword"><Property name="jetty.sslContext.keyStorePassword" deprecated="jetty.keystore.password" default="OBF:1sot1v96\
1saj1v9i1v941sar1v9g1sox"/></Set>

</Configure>
5. jetty-https.xml
<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "http://www.eclipse.org/jetty/configure_9_3.dtd">

<!-- ============================================================= -->
<!-- Configure a HTTPS connector.                                  -->
<!-- This configuration must be used in conjunction with jetty.xml -->
<!-- and jetty-ssl.xml.                                            -->
<!-- ============================================================= -->
<Configure id="sslConnector" class="org.eclipse.jetty.server.ServerConnector">
  <Call name="addIfAbsentConnectionFactory">
    <Arg>
      <New class="org.eclipse.jetty.server.SslConnectionFactory">
        <Arg name="next">http/1.1</Arg>
        <Arg name="sslContextFactory"><Ref refid="sslContextFactory"/></Arg>
      </New>
    </Arg>
  </Call>

  <Call name="addConnectionFactory">
    <Arg>
      <New class="org.eclipse.jetty.server.HttpConnectionFactory">
        <Arg name="config"><Ref refid="sslHttpConfig" /></Arg>
        <Arg name="compliance"><Call class="org.eclipse.jetty.http.HttpCompliance" name="valueOf"><Arg><Property name="jetty.http.compliance\
" default="RFC7230"/></Arg></Call></Arg>
      </New>
    </Arg>
  </Call>
</Configure>

Embedded Jetty server to support https - hard coded Server Parameters

/*
         Create a basic jetty server object without declaring the port. Since
         we are configuring connectors directly we'll be setting ports on
         those connectors.
        */
        Server server = new Server();
        // gracefully shutdown
        server.setStopAtShutdown(true);

        /*
          SSL Context Factory for HTTPS
          SSL requires a certificate so we configure a factory for ssl contents
          with information pointing to what keystore the ssl connection needs
          to know about. Much more configuration is available the ssl context,
          including things like choosing the particular certificate out of a
          keystore to be used.
        */
        SslContextFactory sslContextFactory = new SslContextFactory();
        sslContextFactory.setKeyStorePath(keystoreFile.getAbsolutePath());
        // Fixed me: password as incoming parameter
        sslContextFactory.setKeyStorePassword("OBF:1sot1v961saj1v9i1v941sar1v9g1sox");

/*
          HTTPS Configuration
          HttpConfiguration is a collection of configuration information
          appropriate for http and https. The default scheme for http is
          <code>http</code> of course, as the default for secured http is
          <code>https</code> but we show setting the scheme to show it can be
          done. The port for secured communication is also set here.
          On this HttpConfiguration object we add a SecureRequestCustomizer
          which is how a new connector is able to resolve the https connection
          before handing control over to the Jetty Server.
        */
        HttpConfiguration https_config = new HttpConfiguration();
        https_config.setSecureScheme("https");
        https_config.setSecurePort(port);
        https_config.setOutputBufferSize(32768);

        SecureRequestCustomizer src = new SecureRequestCustomizer();
        src.setStsMaxAge(2000);
        src.setStsIncludeSubDomains(true);
        https_config.addCustomizer(src);

        /*
          HTTPS connector
          We create a second ServerConnector, passing in the http configuration
          we just made along with the previously created ssl context factory.
          Next we set the port and a longer idle timeout.
        */
        ServerConnector https = new ServerConnector(server,
                                                    new SslConnectionFactory(sslContextFactory,HttpVersion.HTTP_1_1.asString()),
                                                    new HttpConnectionFactory(https_config));
        https.setPort(port);
        https.setIdleTimeout(500000);

        // Add HTTPS connector to server
        server.addConnector(https);