10 Steps to Centralize Web App Logs with Graylog

Configuring Your Graylog Server

1. Log in as ‘admin’ to open your Graylog v1.3 Admin Console; typically: http://mygraylogserver:9000/
2. Create a new input (socket) where your App is going to establish connection against Graylog server. As best practice, we are choosing TCP transport for GELF (Graylog Extended Log Format, our transport serialization) since it provides nice features over UDP—such as connection based functionality and transport encryption via TLS: System>Inputs>GELF TCP>Launch New Input Choose a title and connection port for your new input – make sure the port is free and not being assigned to any other process in your Graylog server: Leave the rest of the values unchanged, scroll down and click Launch: After launching your new input, it starts automatically:
3. Now that you have a transport and port ready to receive log messages, you need to tell Graylog where those must be centralized—a Stream suits this purpose: Streams>Create New After saving your new stream, it is time to edit a rule (filter) so your messages can be properly routed: In section 2, leave the first option on and click Add stream rule: Save the source rule as a match exactly type: Verify your new rule appears at the bottom and click I’m done:
4. Start your new stream:
5. Send a message from your application and verify that it shows up in your Graylog stream: Streams>Backend Services (PHP)

Sending Messages from Your Web App

Send Messages to Graylog from a PHP Web App

1. The way we want to log into Graylog is to use the Graylog Extended Log Format (GELF), which can be easily integrated with PHP using the library https://github.com/bzikarsky/gelf-php by Benjamin Zikarsky. It is available on his latest version on Packagist so it can be installed with Composer on your application: composer require graylog2/gelf-php
2. As a regular Composer application we just need to include it:

   require_once __DIR__ . '/../vendor/autoload.php';

3. Gelf-php supports many transport libraries depending of the configuration of our Graylog installation, such as UDP, TCP, HTTP, etc. Given we’re using TCP in our installation, we use the class Gelf\Transport\TcpTransport and just set up the server and port:

   $transport = new Gelf\Transport\TcpTransport("mygraylogserver", 9000);

4. Now, we set up our message to be logged. In this case, we take as an example a cron job that registers leads into our platform, and in case of failure we want to know the cause. Use ShortMessage to define the problem, and the FullMessage field to log the payload response we received from our request. We also have a custom field to log our request parameters, using the method setAdditional:

   $message = new Gelf\Message();
   $message->setShortMessage("Lead couldn’t be created")
       ->setLevel(\Psr\Log\LogLevel::ALERT)
       ->setFullMessage($response); //{“success”:”false”, code:”lead_duplicated”}
     ->setAdditional(‘request',$request); //{“email”:“existing_lead@mail.com”}

5. Finally we set up the host.

     $message ->setHost("portal.backendService");
     $message->setAdditional('component',’Leads/API/create’);

<?php
  require_once __DIR__ . '/../vendor/autoload.php';
$transport = new Gelf\Transport\TcpTransport("mygraylogserver", 9000);
$message = new Gelf\Message();
$message->setShortMessage("Lead couldn’t be created")
    ->setLevel(\Psr\Log\LogLevel::ALERT)
    ->setFullMessage($response) //{“success”:”false”, code:”lead_duplicated”}
  ->setAdditional('request',$request); //{“email”:“existing_lead@mail.com”}
$message ->setHost("portal.backendService");
  $message->setAdditional('component','Leads/API/create');
?>

Send Messages to Graylog from a Java Web App

1. Create a Web Project featuring Maven structure: Go to File>New>Project>MyEclipse>Java Enterprise Projects>Web Project and complete the wizard as shown below.

• Add JSF to the Web Project from the MyEclipse workspace (We love that MyEclipse lets you add JSF facets directly from the context menu!!) Right-click on the graylog-test project, select MyEclipse>Project Facets>Install JavaServer Faces Facet, and then complete the wizard as shown below. Install JavaServer Faces Facet wizard-screen 1: Complete as shown and click Next.Install JavaServer Faces Facet wizard-screen 2: Complete as shown and click Next. Install JavaServer Faces Facet wizard-screen 3: Complete as shown and click Finish.
• Set up log4j. We are going to use Gelfj appender to tell Log4j to send messages to our Graylog server. Let’s add Maven dependencies: Right-click on the graylog-test project, select Maven>Add Dependency and complete the following steps. Add details for Log4j libraries as shown and click OK.  Right-click on the graylog-test project and select Maven>Add Dependency to add the Gelfj libraries: Add details for Gelfj libraries as shown and click OK.Right-click on graylog-test>WebRoot>WEB-INF and select New>File. Type the file name as shown and click Finish. Now let’s do some cool stuff! Let’s send messages to our Graylog server and our Java application without implementing customized Graylog classes. Ready? Enter the following configuration in our new log4j.properties file so we can send INFO messages to Graylog and DEBUG messages to the Web App’s log file:

  # Turn off stdout 
  log4j.rootLogger=OFF
  
  # Graylog appender (Graylog server = monitor)
  log4j.graylog=INFO, graylog
  log4j.appender.graylog=org.graylog2.log.GelfAppender
  log4j.appender.graylog.graylogHost=tcp:monitor
  log4j.appender.graylog.graylogPort=12203
  log4j.appender.graylog.originHost=BackendService
  log4j.appender.graylog.layout=org.apache.log4j.PatternLayout
  log4j.appender.graylog.facility=gelf-java-appender
  log4j.appender.graylog.additionalFields={'environment': 'DEV', 'application': 'JavaApp'}
  log4j.appender.graylog.extractStacktrace=true
  log4j.appender.graylog.addExtendedInformation=true
  
  # Application appender (log file)
  log4j.appLogger=DEBUG, appLogger
  log4j.appender.appLogger=org.apache.log4j.RollingFileAppender
  log4j.appender.appLogger.MaxFileSize=15MB
  log4j.appender.appLogger.MaxBackupIndex=20
  log4j.appender.appLogger.File=${catalina.base}/logs/graylog-test.log
  log4j.appender.appLogger.layout=org.apache.log4j.PatternLayout
  log4j.appender.appLogger.layout.ConversionPattern=%d %5p [%t] (%C:%L) - %m%n
  
  log4j.category.graylogLogger=INFO, graylog
  log4j.additivity.graylog=false
  
  log4j.category.webAppLogger=DEBUG, appLogger
  log4j.additivity.appLogger=false

  Now, let’s initialize out Log4j Engine with this class:

  package com.myeclipse.graylog;
  
  import javax.servlet.http.HttpServlet;
  import org.apache.log4j.PropertyConfigurator;
  
  public class Log4jInit extends HttpServlet {
  	// Make sure to generate an unique UID
  private static final long serialVersionUID = 2351548619095521786L;
  
  	public void init() {
  		String prefix = getServletContext().getRealPath("/");
  		String file = getInitParameter("logFile");
  		if (file != null) {
  			PropertyConfigurator.configure(prefix + file);
  		}
  	}
  }

  Finally, let’s make Log4j load at Web App startup by updating our graylog-test/WebRoot/WEB-INF/web.xml file:

    <servlet>
      <servlet-name>LoggerServlet</servlet-name>
      <servlet-class>com.myeclipse.graylog.Log4jInit</servlet-class>
      <init-param>
          <param-name>logFile</param-name>
          <param-value>/WEB-INF/log4j.properties</param-value>
      </init-param>
      <load-on-startup>1</load-on-startup>
    </servlet>
  

• Create a JSF page to send a log message to Graylog:

  Managed Bean

  package com.myeclipse.graylog;
  
  import java.io.Serializable;
  
  import javax.faces.bean.ManagedBean;
  import javax.faces.bean.ViewScoped;
  
  import org.apache.log4j.Logger;
  
  @ManagedBean
  @ViewScoped
  public class Recovery implements Serializable {
  	private static final long serialVersionUID = -6989213978614032835L;
  	private static final Logger WebAppLogger = Logger.getLogger("webAppLogger");
  	private static final Logger GrayLogLogger = Logger.getLogger("graylogLogger");
  	
  	private String email;
  	
  	public Recovery() {
  		this.init();
  	}
  	
  	private void init() {
  		WebAppLogger.debug("Starting WebApp...");
  		this.email = new String();
  	}
  
  	public String passwordReset() {
  		// This message will NOT be sent to our Graylog server 
  		WebAppLogger.debug("Received UID = " + this.email);
  		
  		// This message will be sent to our Graylog server through our 
  		// defined 'graylog' appender in 'log4j.properties' file  
  		GrayLogLogger.info("A password reset has been requested by: " + this.email);
  		
  		return null;
  	}
  	
  	public String getEmail() {
  		return email;
  	}
  
  	public void setEmail(String email) {
  		this.email = email;
  	}
  }
  

  JSF Page

  <?xml version="1.0" encoding="UTF-8"?>
  <!DOCTYPE html>
  
  <html xmlns="http://www.w3.org/1999/xhtml"
  	xmlns:ui="http://xmlns.jcp.org/jsf/facelets"
  	xmlns:h="http://xmlns.jcp.org/jsf/html"
  	xmlns:f="http://xmlns.jcp.org/jsf/core"
  	xmlns:c="http://xmlns.jcp.org/jsp/jstl/core"
  	xmlns:fn="http://xmlns.jcp.org/jsp/jstl/functions" lang="en">
  	<h:head>
  		<title>JSF Test Page For Graylog</title>
  		<meta name="keywords" content="graylog,log4j,gelfj"/>
  		<meta name="description" content="e-mail UID will be logged into Graylog"/>
  		<meta name="content-type" content="text/html; charset=UTF-8"/>
      </h:head>
  
  	<h:body>
  		<p>
  			<h:form id="recoveryForm">
  				<h:outputText value="Enter your e-mail address: " />
  				<h:inputText value="#{recovery.email}" size="32" />
  				<br />
  				<h:commandButton type="submit" value="Request Password" 
  action="#{recovery.passwordReset}" />
  			</h:form>
  		</p>
  	</h:body>
  </html>
  

• Finally, let’s test our Java App by entering an e-mail address so we can locate our Web App log message in Graylog stream: http://localhost:8080/graylog/index.xhtml Tailing our Web App’s log file to locate our DEBUG messages (graylog-test.log): On our Graylog Server admin console, locate our INFO messages: Streams>Backend Services

Conclusion

Let Us Hear from You!