Monday, 31 October 2016

This week 17/2016

SmartParam (v.1.1.2) is a powerful framework to parametrise an application.

1. Source of configuration
It is possible to load configuration from text file, database or create it dynamically.  Framework supports directly H2, MySql and Postgesql databases.


example of configuration file:

{
  name: "simpleCase",  inputLevels: 2,  cachable: true,  nullable: true,  arraySeparator: ";",
  levels: [
    {name: "registrationDate", type: "date",   matcher: "between/ie" },     
    {name: "name",                     type: "string" }, 
    {name: "value",                     type: "integer" }, 
    {name: "value2",                   type: "integer" }
  ]
}
registrationDate;name;value;value2
*: 2013-12-01;Adam;20*: 2013-12-01;Monia;5
 


2. Number of input/output parameters
At the beginning it is required to define structure of parameters:
  • name - name of configuration,
  • inputLevels - count of input parameters, which are used to wind exact output parameters,
  • nullable - by default it is false and if any rule match input parameters, it throws exception
  • levels - defines types of input and output parameters.
3. Supported parameters
Framework supports simple types of data, however it is possible to define your own type. There are two ways to do that: do converter or  type holder.
Type holder ex.
@ParamType("localdate")
public class LocalDateType implements Type<LocalDateHolder> {
.....
}
 
public class LocalDateHolder extends AbstractValueHolder {
    private final LocalDate date;
    public LocalDateHolder(LocalDate date) {
        this.date = date;    }
....
} 

4. Matcher
Framework provides all needed matchers but there is possible to create your own matcher.
ex.
@ParamMatcher("mymatcher")
public class MyMatcher implements Matcher {

    @Override    public <T extends ValueHolder> boolean matches(String s, String s1, Type<T> type) {

        return s.toString().equals(s1.toString());    }
}

One of useful matcher is BetweenMatcher. It supports defining a ranges of dates, numbers and defines default value by *

5. Other Notices
Framework presents itself great but project looks to be extinct.

Saturday, 22 October 2016

This week 16/2016

How works a synchronisation of some object in a nutshell.

My goal is to synchronise a block of code by some string which is a parameter of a function.

At the begging I wasn't sure how synchronisation mechanism works. Is it using equals and hash code to compare object or memory address where object is allocated.
So synchronisation doesn't use equal method but memory address where object is allocated to.

So how to synchronise part of code when I have many instances of the same object (ex. string)?

A solution is to use some container for synchronized objects and retrieves instance of  that object to use it to synchronization. In this case the best choose is map where it is possible to find object by key and get its value or if it not exists yet, add it.

So we need some kind of cache but how to manage this cache? We need only object instance when the synchronisation block is executed.
I think the easiest way on a single jvm is to use a weak reference. In this case I create a cache manager with WeakHashMap. The manager has a synchronized method findOrAdd and returns an instance of string object included to map.
What is important the WeakHashMap use weak reference only for key object so it is required to put as a value a WeakReference instance with my object.

Map map = new WeakHashMap();
map.put(myString, new WeakReference(myString));
Now, if instance of myString hasn't any reference to any other object or is not used to synchronisation, it will be removed from memory so removed from map as well.

Saturday, 8 October 2016

This week 15/2016

Protractor framework.

The Protractor (v4.0.x) is perfect tool to testing Angular JS application. The main advantage is that it transparently supports asynchronous tasks. You don't care of timeout for Angular asynchronous operation, Protractor is synchronising itself for you. Big advantage is that framework can make snapshot of screen and save it to ex. PNG file.

Saturday, 17 September 2016

This week 14/2016

A few weeks ago I started doing a Angular 2 tutorial. Angular2 is extremely different to first beta version. I chose coding in TypeScript which is recommended language. It is possible to coding in JavaScript or Dart as well but it is less popular.

In my case I chose TypeScript which is compiled to JavaScript and then interpreted by browser. Writing in TypeScript is nice for developer with Java grounds.

Angular 2 defines 4 types of objects: Component, Service, Module and Pipe. 

Before the first final release was published, the concept how to bootstrap application had changed. As I remember, first version of Angular2 allows to execute project without using main module. Later it was changed.
My experience is that the most difficult thing is to match collaborative versions of libraries. I spend much time on selecting exact version of libraries to run some easy application.

And one more think. To create and build production version of code I recommend to use angular-cli.



Saturday, 27 August 2016

This week 13/2016

This week I was creating a functionality. An independent service to export large part of data to MS Excel file and part of service which retrieve data from database. It is obvious that xls format can contain about 65 thousand of rows, so I decided to use xlsx format which I thought it is unlimited but about this it will be later. My requirement was to export from database to excel a large set of data and not kill application. 

First of all I focused on the output. The solution was to not use a XSSFWorkbook but SXSSFWorkbook. In my application currently I use old version of Apache POI v3.7 and there isn't implemented SXSSFWorkbook so in this case there is impossible to solve my problem. SXSSFWorkbook is available from v3.8.
However what I could do after I upgrade a libraries? I checked and it is possible to export huge part of data using less then 64MB heap memory.  The SXSSFWorkbook implementation can save simple data in a stream. Process of creating file is split into two phases. In the first phase implementation is saving processed data into temporary file (on linux it is /tmp/.... file). In the second phase temporary xml file is compressed with additional files containing styles and other information into final file.

By the way I found out that xlsx is not unlimited and every sheet can have maximum a little more then one million rows (2^20) and about 16 thousand columns (2^14).

After I had found out how to export large volume of data to xlsx I looking for solution how to retrieve data from database row by row. I'd like to separate input from output service. I created interface of DataProvider and injected there a RowMapper and other types used in NamedParameterStatement's query method but it doesn't work. Finally I used a ScrollableResultSet with Forward option and limitation of retrieved data at once and it works.


Saturday, 20 August 2016

This week 12/2016

I watched a few presentation about jvm and a garbage collection. Today I will write about topics which are new for me.

I didn't know that:...

1. JVM have a lot of parameters (a few hundreds) and some of them are manageable. It is possible to change their state in runtime (about hundred).

2. Bytecode is compiled by JIT into processor code in runtime but there are a few types of compiled code, depending of:
- free memory for compiled code - every platform have other size of memory
- frequency of usage - often used part of code is better optimised then used onece,
- count of processors and cores,
- complication of code's part.

3. Log4J retrieve logged line of code by dumping a stack trace. It is an expensive operation.
4. New one GC called G1 solve a fragmentation of data by splitting memory area into blocks.

Saturday, 6 August 2016

This week 11/2016

This week I was interested in collecting statistic from my application. I have never been doing that and I think a few times, it could be helpful to resolve some problems. Currently there is other motivation - from it depends my half year premium.

Ok, so I have my Java application and what's next?
I can measure ex. how is used cache, heap memory, CPU and etc.
How can I get this information? 
I can log it to file or other adapter or I can serve it by JMX MBean. Logging to file takes hard drive memory and it can be weight. JMX works as JMS service but it's allow to change some application parameters and is light.

Anyway I started from log statistic in file. What I did?

JVM:
Everything about jvm memory and threads is in class ManagementFactory with static methods. Only you have to dump it to log. MBeans are by default registered in JMX Server too.
I didn't find counter of blocked threads and not dead locked, so I prepared my own method to update MBean as below.

        ThreadInfo[] infos = ManagementFactory.getThreadMXBean().dumpAllThreads(true, true);
        int blocked = 0;
        for(ThreadInfo ti : infos){
            if(Thread.State.BLOCKED.equals(ti.getThreadState())){
                blocked++;
            }
        }
        mbean.setBlocked(blocked);


Hibernate statistic:
I had to add in configuration parameter hibernate.generate_statistics and than I could get Statistics interface and enable collecting statistic in SessionFactory.

statistics = sessionFactory.getStatistics();
statistics.setStatisticsEnabled(true);
All values are available in Statistics objec.

If you need JMX you have to registry Statistic object as MBean in JMX Server.



EHCache statistic
To get EHCache statistics it is required add statistics attribute to every cache container definition.
statistics="true"

 Then it is possible to get statistics from every cache managers.
I noticed when there is a few cache factories, it is needed to set them not shared. Only cache shared with hibernate should be shared. Otherwise I couldn't get to cache manager for hibernate. I saw only one which I defined in spring configuration.

When I had two different configurations, finally two cache managers. I had to share only one which was based on hibernate ehcache file. Others shouldn't be shared otherwise I couldn't get reference to hibernate cache manager.


By the way I will write how to registry your own MBean on your JVM. What you need is to get JMX service and register your MBean. MBean class have to implement interface with postfix MBean. All method included in interface are available from JMX console. Getters presents values, setters change it and other methods can be executed from jconsole. Example code is shown below.

    MyHello mbean = new MyHelloMBean();
    MBeanServer mbs =  ManagementFactory.getPlatformMBeanServer();
    ObjectName name = new ObjectName("com.example:type=Hello");
    mbs.registerMBean(mbean, name);