This week 13/2016

This week I was creating a functionality. An independent service to export large part of data to MS Excel file and part of service which retrieve data from database. It is obvious that xls format can contain about 65 thousand of rows, so I decided to use xlsx format which I thought it is unlimited but about this it will be later. My requirement was to export from database to excel a large set of data and not kill application. 

First of all I focused on the output. The solution was to not use a XSSFWorkbook but SXSSFWorkbook. In my application currently I use old version of Apache POI v3.7 and there isn't implemented SXSSFWorkbook so in this case there is impossible to solve my problem. SXSSFWorkbook is available from v3.8.
However what I could do after I upgrade a libraries? I checked and it is possible to export huge part of data using less then 64MB heap memory.  The SXSSFWorkbook implementation can save simple data in a stream. Process of creating file is split into two phases. In the first phase implementation is saving processed data into temporary file (on linux it is /tmp/.... file). In the second phase temporary xml file is compressed with additional files containing styles and other information into final file.

By the way I found out that xlsx is not unlimited and every sheet can have maximum a little more then one million rows (2^20) and about 16 thousand columns (2^14).

After I had found out how to export large volume of data to xlsx I looking for solution how to retrieve data from database row by row. I'd like to separate input from output service. I created interface of DataProvider and injected there a RowMapper and other types used in NamedParameterStatement's query method but it doesn't work. Finally I used a ScrollableResultSet with Forward option and limitation of retrieved data at once and it works.

This week 12/2016

I watched a few presentation about jvm and a garbage collection. Today I will write about topics which are new for me.

I didn't know that:...

1. JVM have a lot of parameters (a few hundreds) and some of them are manageable. It is possible to change their state in runtime (about hundred).

2. Bytecode is compiled by JIT into processor code in runtime but there are a few types of compiled code, depending of:
- free memory for compiled code - every platform have other size of memory
- frequency of usage - often used part of code is better optimised then used onece,
- count of processors and cores,
- complication of code's part.

3. Log4J retrieve logged line of code by dumping a stack trace. It is an expensive operation.

4. New one GC called G1 solve a fragmentation of data by splitting memory area into blocks.

This week 11/2016

This week I was interested in collecting statistic from my application. I have never been doing that and I think a few times, it could be helpful to resolve some problems. Currently there is other motivation - from it depends my half year premium.

Ok, so I have my Java application and what's next?
I can measure ex. how is used cache, heap memory, CPU and etc.

How can I get this information? 

I can log it to file or other adapter or I can serve it by JMX MBean. Logging to file takes hard drive memory and it can be weight. JMX works as JMS service but it's allow to change some application parameters and is light.

Anyway I started from log statistic in file. What I did?

JVM:
Everything about jvm memory and threads is in class ManagementFactory with static methods. Only you have to dump it to log. MBeans are by default registered in JMX Server too.

I didn't find counter of blocked threads and not dead locked, so I prepared my own method to update MBean as below.

        ThreadInfo[] infos = ManagementFactory.getThreadMXBean().dumpAllThreads(true, true);
        int blocked = 0;
        for(ThreadInfo ti : infos){
            if(Thread.State.BLOCKED.equals(ti.getThreadState())){
                blocked++;
            }
        }
        mbean.setBlocked(blocked);

Hibernate statistic:
I had to add in configuration parameter hibernate.generate_statistics and than I could get Statistics interface and enable collecting statistic in SessionFactory.

statistics = sessionFactory.getStatistics();

statistics.setStatisticsEnabled(true);

All values are available in Statistics objec.

If you need JMX you have to registry Statistic object as MBean in JMX Server.



EHCache statistic
To get EHCache statistics it is required add statistics attribute to every cache container definition.

statistics="true"

 Then it is possible to get statistics from every cache managers.

I noticed when there is a few cache factories, it is needed to set them not shared. Only cache shared with hibernate should be shared. Otherwise I couldn't get to cache manager for hibernate. I saw only one which I defined in spring configuration.

When I had two different configurations, finally two cache managers. I had to share only one which was based on hibernate ehcache file. Others shouldn't be shared otherwise I couldn't get reference to hibernate cache manager.

By the way I will write how to registry your own MBean on your JVM. What you need is to get JMX service and register your MBean. MBean class have to implement interface with postfix MBean. All method included in interface are available from JMX console. Getters presents values, setters change it and other methods can be executed from jconsole. Example code is shown below.

    MyHello mbean = new MyHelloMBean();
    MBeanServer mbs =  ManagementFactory.getPlatformMBeanServer();
    ObjectName name = new ObjectName("com.example:type=Hello");
    mbs.registerMBean(mbean, name);

This week 10/2016

This weekend I got to know a little Groovy (v2.4). The most important for me was, how to start developing. I opened one tutorial and I did step by step every action. Finally to use Groovy in eclipse I had to add resource for my version (Mars) and install the Groovy tools. The resource provide the older version of Groovy, not supporting dynamic invoke. I didn't find out how is in IntelliJ case. Anyway it is possible to download jars manually or by extension in Maven and then you can start implementing in Groovy.

Using Eclipse version of Groovy I tested if it is possible to use Jdk8 lambda expression in Java method and then execute it in Groove. Test passed, code compiled and executed.
Now let's study Groovy syntax and it semantic.

This week 9/2016

It pass a month since last post. I had a break because of my holidays and some other important things. Last week I was at Devoxx conference in Cracow. I'd like to write in a nutshell about some main topics.

1. What does Unsafe class and why there is some cry after it will miss?

Unsafe is a class in com.sun package. It was created for internal use of java classes but after some time it is used by common libraries. The Unsafe allow to manual managing memory usage, ex. creating table with greater index than allow int type. GC doesn't involve this part of memory.

2. Jdk8 - collections and lambda's expressions.

There was shown, how works lazy initialization and processing in collections. 

What gives us lambada expressions.

3. Next releases of Java. Architecture plans.
In Java 9 there is planned standardise an API. They plan to remove close 30 classes from com.sun package. The most problematic is Unsafe class. I mentioned about it role before.
Other main changes are:
- array index will be changed to long type,
- a generic class will save its type in bytecode. It will be possible overloading methods with generic collection.

4. Reactive programming - Rx.Java and Hysterix

Another time there was described Hysterix as a simple and excellent framework to maintenance microservices connections.

5. DDD and bounded domain context. One domain model is too huge and complicated to maintenance it. The best idea is to split it into subdomains connected with some context. Nobody shown a real code of that solution. I always asking me, how to do it with hibernate. Is it at all possible?

6. Angular JS 2
I didn't take a part in that sessions. Anyway sessions described how different is AngularJs 2 from AngularJs 1, pointed the way how to start. In a nutshell Angular 2 was totally changed. Now code is writing in typescript and than is compiled to javascript. As it could be noticed, it is totally other language.

7. Groovy - how to start and difference between syntax in Groovy and Java.
At first it is possible to change Java code file to groovy file and compile it. It will be working.  Groove is slower language, so is not recommended to use on production but it is very useful to create tests. It is possible to basic Java syntax and it will be working to. There is huge support for collections. Anyway last version of Groovy don't support lambda expressions from Java 8. But it is possible to code in Groovy and call Java classes and vice versa and then you can use lambda expressions in Java code. All details about language can be find on page http://groovy-lang.org/style-guide.html
One important notice. Sometimes the same code in Groovy could work otherwise than in Java.

8. JUnit 5 - futures


9. How to pass presentations. Good advice given by programmer for programmers.
Sławomir Sobótka was talking about causes of stage fright. He diagnosed that stage fright is caused by too high self-esteen and fright that it could be reduced.


10. Microservices

This week 8/2016

This week I try to offload a queue of articles I'd like to read. I found some interesting blog about Gradle (v3.x) getting-started-with-gradle. Blog hides you from installig the tool to develop multi-module project. This time I just read multiple articles and I didn't do any examples, so my knowledge is shallow. Anyway I collect here main notices and differences between it and Maven.
In the nutshell the Gradle is much more powerful tool. It left traditional format of a configuration (XML) and took advantages of the closure syntax, so file look much more simple.

In Gradle it is possible to define own tasks and mark dependences - it's memorise me a Makefile. Gradle interprets Groovy language, ex:

from { configurations.compile.collect { it.isDirectory() ? it : zipTree(it) } }

Other advantage is that it can generate test reports and you can separate reports for unit testing and integration testing.

A configuration file is smaller than pom file.

This week 7/2016

This week I finished configure fragments in my simple application. As I mentioned that a filed marked as a number field is associated with a String type of model attribute. If you need to associate your field with other time it is required to create your own binder method.

A long time took me to find out how to make unusual binder method for spinner.  In my case, I needed to bind a view action to a model. I used standard spinner listener to update my model but it wasn't a final solution. The best was to create your own binding method with a special annotations contains used in XML parameters. It is possible to bind more than one parameter at once.

ex.

    @BindingAdapter({"bind:enumElements", "bind:selected"})
    public static <T> void spinnerEnumBinnder(Spinner view, Enum<?>[] enumClass, T selected) {
        if (view.getAdapter() == null) {
            loadEnums(view, enumClass);
        }
        .....
    }

And than it is needed to add in layout XML:

<?xml version="1.0" encoding="utf-8"?>
<layout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:tools="http://schemas.android.com/tools"
    xmlns:bind="http://schemas.android.com/apk/res-auto">
    <data>
        <variable name="linkName" type="org.example.ModelObject" />
    </data>
    <LinearLayout android:orientation="vertical" tools:context="org.example.ParamFragment">
        <Spinner
            android:id="@+id/objectType"
            bind:selected="@{linkName.type}"
            bind:enumElements="@{linkName.deviceTypes}"
            android:layout_gravity="center_horizontal|top"
            android:spinnerMode="dropdown" />
    </LinearLayout>
</layout>