Exploring the logs

As a developers an important part of our job sometimes is to fix problems in the different environments where our applications are deployed. Usually, this means to deal with huge log files to find where errors occur, and their stacktraces to add some context to the problem. The problem is that usually log files are verbose and contain a lot of information.

A couple of useful command to deal with this can be:

  • grep
  • zgrep

Both have the same purpose the only difference it that “grep” works with normal files and “zgrep” works with compressed (.gz) files. Usualy files are compressed due to the logs rotation scheduled in the servers. Both commands have multiple options and flags but, I am going to expose here two flags that have been useful multiple times:

  • -E expr: Allow as to supply a pattern for the search.
  • -C num: Print num lines of leading and trailing output context.
  • –color: Shows the matched information in color in the terminal.

As an example we have:

zgrep --color -E '(Sending email)' myLog.log-20170621.gz
grep --color -E '(Sending email)' myLog.log
grep --color -C 25 -E '(Sending email)' myLog.log

As we can see, obviously, they can be combined.

 

Advertisements
Exploring the logs

Simplifiying SSH

Nowadays we are use to deploy code in the cloud and to have all our machines and servers in cloud environments. All of this, it has even made more important the use of ssh to connect remotely to our servers allocated in the cloud.

I have written multiple times in my console the commands to connect to one server or another but, as every developer, I am lazy and I try to simplify my life. In this case, we can do this with a simple lines in a couple of files:

  • ~/.ssh/config: We are going to configure the machines we want to connect or tunnels we wnat to create.
  • ~/.bashrc or ~/.bash_profile: Create some alias to easily connect to our servers

SSH config file

Server to connect

# MyServer-1
Host                    myServer1
HostName                myserver1.myorg.com
User                    username
IdentityFile            ~/.ssh/myCertificate.pem
PasswordAuthentication  no
StrictHostKeyChecking   no

Create tunnel

# MyServer-1 - myDb
Host                    myServer1Db
HostName                myserver1.myorg.com
User                    username
IdentityFile            ~/.ssh/myCertificate.pem
PasswordAuthentication  no
StrictHostKeyChecking   no
LocalForward            3307 myserver1.myorg.com:3306

Bash Config file

alias myserver1="ssh myServer1"
alias myserver1db="ssh myServer1Db"

Conclusion

After this, it will be enough to connect to our remote servers with executing our aliases in our console. No more remember commands.

 

Simplifiying SSH

Debugging JVM flags

Just a couple of interesting flags when we are executing/debugging a Java web based application.

First one, it is just to change the logging level without modifiying our properties file:

-Dlogging.level.com.wordpress.binarycoders=DEBUG

Second one, it is in case we are using Hibernate as ORM. It will allow us to see the executed SQL queries in the log:

-Dhibernate.show_sql=true

They just need to be added as a “VM Options” when the server is started or the JAR file is run.

Debugging JVM flags

Error Prone

We, as a developers, sometimes, make mistakes or add bugs to our code without realizing. For this reason static analyzers are a handy tool to apply during our builds or during our code verification processes.

One of these tools is Error Prone.

Error Prone is Google’s Java bug detection and static analysis tool. It is integrated into the Java compiler and catches bugs at compile time. It supports plugin checks for project-specific enforcement.

Basically, it is a tool created by Google for code analysis and error detection for the Java language. It is integrated inside the compiler and tries to detect bugs in compilation time.

But, let’s see and example. Imagine we have a program with the next line of code:

String.format("Param A: {}, param B: {}, param C: {}", paramA, paramB, paramC);

Obviously, it is not correct and the error comes from, maybe, a transformation between a previous log message to a different kind of message. The compiler is not going to complain because it is a string message and it is not a syntax error. But, the truth is there is an error.

When we try to compile the program with Error Prone, we are going to receive a compilation error message like this:

error: [FormatString] extra format arguments: used 0, provided 3
String.format("Param A: {}, param B: {}, param C: {}", paramA, paramB, paramC);
             ^
(see http://errorprone.info/bugpattern/FormatString)

We can see clearly and without any doubts there is an error. Even, a link to the error description is provided.

The proper code should be:

String.format("Param A: %s, param B: %s, param C: %s", paramA, paramB, paramC);

The easiest way to start using the tool, it is to add the maven plugin to our pom.xml file:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-compiler-plugin</artifactId>
    <version>3.3</version>
    <configuration>
        <compilerId>javac-with-errorprone</compilerId>
        <forceJavacCompilerUse>true</forceJavacCompilerUse>
        <source>8</source>
        <target>8</target>
    </configuration>
    <dependencies>
        <dependency>
            <groupId>org.codehaus.plexus</groupId>
            <artifactId>plexus-compiler-javac-errorprone</artifactId>
            <version>2.8.1</version>
        </dependency>
        <dependency>
            <groupId>com.google.errorprone</groupId>
            <artifactId>error_prone_core</artifactId>
            <version>2.0.19</version>
        </dependency>
    </dependencies>
</plugin>

For more options, just go to the installation instructions page.

The project is open source and you can see all the code in the official repository: error-prone.

I am not saying that it is going to solve all your problems but, at least, it is another tool to increase our code quality and avoid silly mistakes.

Error Prone

Customized date format in JAXB

Nowadays, it is very common to work with JAXB to help us to marshal and unmarshal objects into XML or XML into objects. These objects can be simple object or objects created by us if we use the correct annotations. It is clear that JAXB is very useful but, sometimes, it is not enough with the default values the API is offering us.

Recently, we had a problem with the date format. The external system where we were trying to send information was expecting a very concrete pattern for the date objects in the XML, and the default format JAXB was offering us was not correct. After test a couple of solutions, we find one that it was working very good and it was quite simple to implement.

The idea was to rewrite the adapter with the desired format and after that, using the correct annotation, tell JAXB be the correct format. Easier than it sounds.

The first step is to write our adapter object extending the object XmlAdapter. In there, we can define the exact format we want to use when the marshalling operation is performed.

package com.wordpress.binarycoders.services.jaxb.utils;

import java.text.DateFormat;
import java.text.SimpleDateFormat;
import java.util.Calendar;
import java.util.GregorianCalendar;

import javax.xml.bind.annotation.adapters.XmlAdapter;
import javax.xml.datatype.DatatypeFactory;
import javax.xml.datatype.XMLGregorianCalendar;

public class DateAdapter extends XmlAdapter<String, XMLGregorianCalendar> {

    private DateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd");

    @Override
    public String marshal(XMLGregorianCalendar v) throws Exception {
        Calendar calendar = v.toGregorianCalendar();
        
        return dateFormat.format(calendar.getTime());
    }

    @Override
    public XMLGregorianCalendar unmarshal(String v) throws Exception {
        
        GregorianCalendar c = new GregorianCalendar();
        c.setTime(dateFormat.parse(v));
        
        return  DatatypeFactory.newInstance().newXMLGregorianCalendar(c);
    }

}

This example is using a XMLGregorianCalendar object, but this can be done with any date object.

The second step, it is to indicate JAXB that this adapter should be used instead of the default one. We do this with the annotation @XmlJavaTypeAdapter. Something like this:

@XmlSchemaType(name = "date")
@XmlJavaTypeAdapter(DateAdapter.class)
protected XMLGregorianCalendar date;

That´s all. Now, when the marshaling operation is performed, we should obtain the desired date format in our XMLs.

See you.

Customized date format in JAXB

Checksum calculation

Today, we have another little code snipped. Sometimes, we need to transfer files between systems, and in these cases, it is usually interesting to have some kind of system to check the file content integrity. In these cases, it is when a checksum can be very useful. Here we have a little implementation done in Java. Simple to write, simple to use.

package com.wordpress.binarycoders.checksum;

import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.util.Formatter;

public class CheckSum {

    private static final String MD5 = "MD5";
    private static final String SHA1 = "SHA-1";
    
    public static String md5CheckSum(byte[] content) throws NoSuchAlgorithmException {
        MessageDigest md = MessageDigest.getInstance(MD5);
        
        return byteArray2Hex(md.digest(content));
    }
    
    public static String sha1CheckSum(byte[] content) throws NoSuchAlgorithmException {
        MessageDigest md = MessageDigest.getInstance(SHA1);
        
        return byteArray2Hex(md.digest(content));
    }
    
    private static String byteArray2Hex(final byte[] hash) {
        String result = "";
        
        try (Formatter formatter = new Formatter()) {
            for (byte b : hash) {
                formatter.format("%02x", b);
            }
            
            result = formatter.toString();
        }
        
        return result;
    }
}

See you.

Checksum calculation

Image URL to byte array

Today, I only have a little code snippet to take a URL with an image and transform it in a byte array in Java 8, the image, not the URL. It is something very simple, but sometimes, it is very useful to have this kind of snippets in a place where we can find it easily.

package com.wordpress.binarycoders.image.recovery;

import java.io.ByteArrayOutputStream;
import java.io.InputStream;
import java.net.URL;
import java.util.Arrays;
import java.util.logging.Level;
import java.util.logging.Logger;

public class ImageRecover {
    
    public byte[] recoverImageFromUrl(String urlText) throws Exception {
        URL url = new URL(urlText);
        ByteArrayOutputStream output = new ByteArrayOutputStream();
        
        try (InputStream inputStream = url.openStream()) {
            int n = 0;
            byte [] buffer = new byte[ 1024 ];
            while (-1 != (n = inputStream.read(buffer))) {
                output.write(buffer, 0, n);
            }
        }
    
        return output.toByteArray();
    }
}

See you.

Image URL to byte array