Posts

Improving performance of Kafka Producer

Image
  As we know, Kafka uses an asynchronous publish/subscribe model. While our producer calls the send() command, the result returned is a future. That future offers methods to check the status of the information in the process. Moreover, as the batch is ready, the producer sends it to the broker. Basically, the broker waits for an event, then, receives the result, and further responds that the transaction is complete. For latency and throughput, two parameters are particularly important for Kafka performance Tuning: Batch Size Instead of the number of messages, batch.size measures batch size in total bytes. That means it controls how many bytes of data to collect, before sending messages to the Kafka broker. So, without exceeding available memory, set this as high as possible. Make sure the default value is 16384. However, it might never get full, if we increase the size of our buffer. On the basis of other triggers, such as linger time in milliseconds, the Producer sends the informa...

Decorator design pattern in Java

Image
  The intent of the Decorator Design Pattern is to attach additional responsibilities to an object dynamically. Decorators provide a flexible alternative to sub-classing for extending functionality. The Decorator Pattern is used to extend the functionality of an object dynamically without having to change the original class source or using inheritance. This is accomplished by creating an object wrapper referred to as a Decorator around the actual object. The Decorator object is designed to ave the same interface as the underlying object. This allows a client object to interact with the Decorator object in exactly the same manner as it would with the underlying ctual object. The Decorator object contains a reference to the actual object. The Decorator object receives all requests (calls) from a client. In turn, it forwards these calls to the underlying object. The Decorator object adds some additional functionality before or after forwarding requests to the underlying object. This ...

CRUD Goes Even Easier With JPABuddy

Image
Create, Read, Update and Delete are the four basic operations of persistence storage. We can say these operations collectively as an acronym CRUD. These operations can be implemented in JPA. JPA is a standard for ORM. It is an API layer that maps Java objects to the database tables.  ORM stands for Object Relational Mapping. It converts data between incompatible type systems in object-oriented programming languages.  JPA Buddy is an IntelliJ IDEA plugin that helps developers work efficiently. JPA Buddy is a tool that is supposed to become your faithful coding assistant for projects with JPA and everything related. It is an advanced plugin for IntelliJ IDEA intended to simplify and accelerate everything related to JPA and surrounding mainstream technology. In fact, you can develop an entire CRUD application or a simple microservice by spending nearly zero time writing boilerplate code. The video demonstrates the features of JPA Buddy by creating a simple CRUD application from ...

Multi-threaded Apache Kafka Consumer

Image
Why do we need multi-thread consumer model?  Suppose we implement a notification module which allow users to subscribe for notifications from other users, other applications. Our module reads messages which will be written by other users, applications to a Kafka clusters. In this case, we can get all notifications of the others written to a Kafka topic and our module will create a consumer to subscribe to that topic.   Everything seems to be fine at the beginning. However, what will happen if the number of notifications produced by other applications, users is increased fast and exceed the rate that can be processed by our module?   All the messages/notifications that haven’t been processed by our module, are still in the Kafka topic. However, things get more danger when the number of messages is too much. Some of them will be lost when the retention policy is met (Note that Kafka retention policy can be time-based, partition size-based, key-based). And more im...

Automate task with Jenkins and jt400 on IBMi

Image
Automation is a key component for making your life easier. There are dozens of regular, boring tasks that can be automated in order to make your life easier.  The need for task automation has been steadily growing in recent years. One of the reasons behind this is that automation can free up 30% of the time you spend working on daily tasks! What Is Task Automation? Task automation is the use of tools or software to automate repetitive, manual tasks. What Are the Benefits of Task Automation? There are way more advantages to task automation than just reducing manual effort!  There are lots of different tools on the market today that make task automation as easy as possible. But the most popular is Jenkins. For Task Automation on IBMi use Jenkins and the IBM Toolbox for Java CommandCall object to send commands to the server on IBMi.  All you need is jt400 library and a simple groovy script defined in external source control repositories and loaded into existing Pipel...

Password Encryption Decryption in Java with Java 9+ modules

Image
  Java AES Encryption and Decryption Advanced Encryption Standard is one of the most popular encryption algorithms. It is an asymmetric encryption algorithm and more secure. AES is generally used for securing sensitive information so we can say that is enough secure. Before AES most of the organizations used the DES( Data Encryption Standard) algorithm which is not secure and most of the organizations considered that highly insecure. Feature of AES AES is the replacement for DES. Some features of AES are listed below: Symmetric key symmetric block cipher 128-bit data, 128/192/256-bit keys Stronger and faster than Triple-DES Provide full specification and design details Software implementable in C and Java Below is the utility class which will be used for encryption and decryption in Java applications. Java Modules Starting with Java 9 a new concept was introduced: modules. Java modules represent a more powerful mechanism to organize and aggregate packages. A Java module is a way ...

JShell to Improve Your Java Skills

Image
  The Java Shell Tool or in short JShell is a major addition in Java 9.  JShell is quite late to the party, as scripting languages like Python and Node introduced similar utilities years ago, and JVM languages like Scala, Clojure, and Groovy followed in their footsteps. But better late than never. Before the introduction of JShell, there was no way one could do REPL (Read-Evaluate-Print-Loop) in Java. A REPL is a simple, interactive programming environment that takes user input (Read), executes the user command (Evaluate), returns the result to the user (Print) and waits for the next user input (Loop). In fact, to even print a single statement in the console it was absolutely necessary to at least have a class and the main method in it. JShell attempts to take away that pain to a certain extent and adds a full-fledged REPL feature in the Java programming language. JShell is an interactive tool that is useful to learn Java language features, APIs and allows quick prototyping, d...