Wednesday, February 28, 2018

SpringBoot Externalization Part 1: Develop Externalization Server

Spring Boot lets you externalize your configuration so that you can work with the same application code in different environments. You can use properties files, YAML files, environment variables, and command-line arguments to externalize configuration. Property values can be injected directly into your beans by using the @Valueannotation, accessed through Spring’s Environment abstraction, or be bound to structured objects through @ConfigurationProperties.


This thread discusses connection of Spring Externalization Config Server and it caches the properties from Git Hub.

The properties files are in YAML format that are stored on GIT. 


 MindTelligentExternalizationApplication Server Configuration:



import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.config.server.EnableConfigServer;

@EnableConfigServer
@SpringBootApplication
public class MindTelligentExternalizationApplication {

public static void main(String[] args) 
{
SpringApplication.run(MindTelligentExternalizationApplication .class, args);
}

}



Package the code with application.yml file:


server:
  port: 8888
spring:
  cloud:
    config:
      server:
        git:
          uri:  https://hsingh@github.com/ymlbranch
          searchPaths: src
          username: hsingh@MindTelligent.com

          password: XXXXXXXX 



Build the Code with Maven:


<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.mindtelligent.github.server</groupId>
    <artifactId>MindTelligentIotConfigurationService</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <packaging>jar</packaging>

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.0.0.RELEASE</version>
        <relativePath /> <!-- lookup parent from repository -->
    </parent>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <java.version>1.8</java.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-config-server</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                <version>Finchley.M8</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

    <repositories>
        <repository>
            <id>spring-milestones</id>
            <name>Spring Milestones</name>
            <url>https://repo.spring.io/libs-milestone</url>
            <snapshots>
                <enabled>false</enabled>
            </snapshots>
        </repository>
    </repositories>


</project>



  • Once this is built. Run the JAR file with the command
java -jar MindTelligentIotConfigurationService-0.0.1-SNAPSHOT.jar

Log on to the browser and validate by:

http://hostname:8888/yml_file/default

You should be able to see the contents on the browser.



Saturday, February 3, 2018

Important Kafka Commands

Important Kafka Commands

MindTelligent has developed an open source Kafka Administration Framework. This framework monitors the health of Kafka And Zookeeper nodes. It also ships a bunch of important Kafka commands.


These commands can be executed from $KAFKA_HOME/bin directory


Command to Set the Kafka Topic retention Period for 10 days:


./kafka-topics.sh --zookeeper localhost:2181 --alter --topic mindtelligent_topic  --config retention.ms=864000000

Command to Set Kafka Kafka partitions to 6 :


./kafka-topics.sh --zookeeper localhost:2181 --alter --topic mindtelligent_topic --partitions 6

Command to View offsets for the Kafka Consumer Group and instances for a consumer group:


./kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group mindtelligent_topic_group


Command to list all Kafka Consumer Groups across all topics:

./kafka-consumer-groups.sh --bootstrap-server localhost:9092 --list


Command to set the Kafka Offset to earliest:


./kafka-consumer-groups.sh --bootstrap-server localhost:9092 --group mindtelligent_topic_group --reset-offsets --to-earliest --topic mindtelligent_topic --execute


Command to set the Kafka Offset to Latest:


./kafka-consumer-groups.sh --bootstrap-server localhost:9092 --group mindtelligent_topic_group --reset-offsets --to-latest --topic mindtelligent_topic --execute

List Kafka Topic:


./kafka-topics.sh --zookeeper localhost:2181 --list



Describe a Kafka Topic:


./kafka-topics.sh --zookeeper localhost:2181 --describe --topic mindtelligent_topic

Purge a Kafka Topic:


./kafka-topics.sh --zookeeper localhost:2181 --alter --topic mindtelligent_topic--config retention.ms=1000

Delete a Kafka Topic: 

./kafka-topics.sh --zookeeper localhost:2181 --delete --topic mindtelligent_topic

Get Number of Messages in a Kafka Toipc:


./kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list localhost:9092 --topic mindtelligent_topic--time -1 --offsets 1 | awk -F ":" '{sum += $3} END {print sum}'


Get the earliest offset still in a topic:

./kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list localhost:9092 --topic mindtelligent_topic  --time -2

Get the latest offset still in a topic:

./kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list localhost:9092 --topic mindtelligent_topic --time -1




Amazon Bedrock and AWS Rekognition comparison for Image Recognition

 Both Amazon Bedrock and AWS Rekognition are services provided by AWS, but they cater to different use cases, especially when it comes to ...