Sunday, June 11, 2023

Title: Uploading CCB and C2M Data files Files from Oracle Object Store to Oracle Utilities Customer Cloud Service (CCS) using Oracle Integration Cloud (OIC)

 Title: Uploading  CCB/C2M Data files from Oracle Object Store to Oracle Utilities Customer Cloud Service (CCS) using Oracle Integration Cloud (OIC)


Introduction:

As the digital landscape continues to evolve, businesses are adopting cloud-based solutions to streamline their operations. In this blog entry, we will explore how to upload files, such as images or documents, from Oracle Object Store to Oracle Utilities Customer Cloud Service (CCS) using Oracle Integration Cloud (OIC). This integration enables you to seamlessly transfer files from Object Store to CCS, ensuring that your blog entries are enriched with relevant and engaging multimedia content.


Prerequisites:

Before you proceed with the integration, ensure you have the following prerequisites in place:


1. An active Oracle Object Store instance with the files you want to upload.

2. Access to Oracle Utilities Customer Cloud Service (CCS) with the necessary permissions to manage files.

3. Access to Oracle Integration Cloud (OIC) with the required permissions to create integrations.


Step 1: Create an Integration in Oracle Integration Cloud (OIC):

1. Log in to your Oracle Integration Cloud account.

2. Create a new integration by selecting "Create Integration" from the OIC dashboard.

3. Provide a name and description for your integration and select the appropriate package.

4. Choose the integration style that best fits your requirements and click "Create."


Step 2: Configure the Source Connection (Oracle Object Store):

1. Within the integration canvas, click on the plus (+) icon to add a connection.

2. Select "Oracle Storage" from the list of available connections.

3. Provide the necessary details to configure the connection, including the Object Store details, authentication method, and credentials.

4. Test the connection to ensure it is set up correctly.


Step 3: Configure the Target Connection (Oracle Utilities Customer Cloud Service - CCS):

1. Similar to Step 2, add a new connection by clicking on the plus (+) icon.

2. Select "Oracle Utilities" from the connection list.

3. Provide the required details to establish the connection, including the CCS instance URL, authentication method, and credentials.

4. Test the connection to verify its functionality.


Step 4: Design the Integration Flow:

1. On the integration canvas, drag and drop the appropriate start activity, depending on your integration style (e.g., "Scheduled Orchestration," "Event-Driven," etc.).

2. Add a "File Read" activity from the component palette and configure it to read the files from the Oracle Object Store.

3. Connect the "File Read" activity to a "File Write" activity representing the target connection to CCS.

4. Configure the "File Write" activity to upload the files to the desired location in CCS.

5. Optionally, you can add additional activities or transformations to modify the file or metadata during the integration flow.

6. Save the integration.


Step 5: Configure the Trigger (if using Event-Driven style):

1. If you chose the "Event-Driven" style, configure the trigger by selecting the appropriate event (e.g., file upload event) that will initiate the integration.

2. Set up the event parameters, such as the Object Store bucket and event filters.

3. Save the trigger configuration.


Step 6: Activate and Monitor the Integration:

1. Activate the integration by clicking the "Activate" button in the top-right corner of the OIC interface.

2. Monitor the integration runs and logs to ensure the successful transfer of files from Object Store to CCS.

3. Test the integration by manually triggering it or performing the event that initiates the integration.


Conclusion:

By integrating Oracle Object Store with Oracle Utilities Customer Cloud Service (CCS) using Oracle Integration Cloud (OIC), you can effortlessly upload blog entry files, enriching your content with multimedia

Monday, June 5, 2023

Creating a partition index in AWS Glue

Creating a partition index in AWS Glue can help speed up queries that rely on specific partition columns. This blog thread illustrates creating a partition index on an AWS Glue table.

Let's assume you have a table called sales_data in AWS Glue, which is partitioned by year, month, and day. If you frequently query the data by year and month, you can create a partition index on these columns to improve performance.

Example: Creating a Partition Index

  1. Set up the Table and Partitions (if not already set): Ensure your table is set up in AWS Glue Data Catalog and is partitioned by year, month, and day.

    python
    import boto3 glue = boto3.client('glue') response = glue.create_table( DatabaseName='my_database', TableInput={ 'Name': 'sales_data', 'PartitionKeys': [ {'Name': 'year', 'Type': 'int'}, {'Name': 'month', 'Type': 'int'}, {'Name': 'day', 'Type': 'int'} ], 'StorageDescriptor': { 'Columns': [ {'Name': 'product_id', 'Type': 'string'}, {'Name': 'quantity', 'Type': 'int'}, {'Name': 'price', 'Type': 'double'} ], 'Location': 's3://my-bucket/sales_data/' } } )
  2. Create a Partition Index: To create a partition index for the year and month columns, use the following example code:

    python
    response = glue.create_partition_index( DatabaseName='my_database', TableName='sales_data', PartitionIndex={ 'Keys': ['year', 'month'], # Specify the columns to index 'IndexName': 'year_month_index' # Name the index } ) print("Partition Index Created:", response)
  3. Verifying the Partition Index: To check that the partition index was created successfully, you can use the get_partition_indexes method:

    python
    response = glue.get_partition_indexes( DatabaseName='my_database', TableName='sales_data' ) print("Partition Indexes:", response['PartitionIndexList'])

Explanation of the Code

  • DatabaseName and TableName specify the database and table in Glue Data Catalog.
  • PartitionIndex includes:
    • Keys: A list of partition columns to index, in this case, ['year', 'month'].
    • IndexName: A unique name for the index, like year_month_index.

Creating this index will allow AWS Glue and any service querying the table, such as Athena, to quickly locate partitions based on year and month, improving performance on queries filtering by these columns.

Sunday, June 4, 2023

Null Pointer at com.sforce.ws.codegen.Compiler.(Compiler.java:48)

When compiling enterprise.wsdl with  force-wsc-58.0.0-uber.jar I was getting the following exception:


$ java -classpath force-wsc-58.0.0-uber.jar com.sforce.ws.tools.wsdlc enterprise-58-0.wsdl enterprise-58-0.0.jar

[WSC][wsdlc.main:72]Generating Java files from schema ...

[WSC][wsdlc.main:72]Generated 2724 java files.

Exception in thread "main" java.lang.NullPointerException

        at com.sforce.ws.codegen.Compiler.<init>(Compiler.java:48)

        at com.sforce.ws.codegen.Generator.compileTypes(Generator.java:137)

        at com.sforce.ws.tools.wsdlc.run(wsdlc.java:129)

        at com.sforce.ws.tools.wsdlc.run(wsdlc.java:163)

        at com.sforce.ws.tools.wsdlc.main(wsdlc.java:72)


To resolve this issue, I pointed to Oracle JDK and that resolved the issue:

$ /c/jdk1-8-0_202/bin/java -classpath force-wsc-58.0.0-uber.jar com.sforce.ws.tools.wsdlc enterprise-58-0.wsdl enterprise-58-0.0.jar
[WSC][wsdlc.main:72]Generating Java files from schema ...
[WSC][wsdlc.main:72]Generated 2724 java files.
[WSC][wsdlc.main:72]Compiled 2728 java files.
[WSC][wsdlc.main:72]Generating jar file ... enterprise-58-0.0.jar
[WSC][wsdlc.main:72]Generated jar file enterprise-58-0.0.jar

Thursday, June 1, 2023

Create HDFS Agent for Oracle Integration Cloud Generation 3

 Create HDFS Agent for Oracle Integration Cloud Generation 3

Oracle Integration 3 is a fully managed, preconfigured environment that gives you the power to integrate your cloud and on-premises applications, automate business processes, develop visual applications, use an SFTP-compliant file server to store and retrieve files, and exchange business documents with a B2B trading.

This code will create an HDFS agent that connects to a Hadoop Distributed File System (HDFS) cluster running on localhost. The agent will be started and then stopped. You can use this code as a starting point for creating your own HDFS agents.

Here are some additional notes about the code:

  • The HdfsAgentProperties class is used to configure the HDFS agent. The properties that can be configured include the HDFS URL, username, and password.
  • The HdfsAgentFactory class is used to create HDFS agents. The factory can be used to create agents that connect to different types of HDFS clusters.
  • The start() and stop() methods are used to start and stop the HDFS agent.


Code Snippet

Code snippet
import com.oracle.integration.cloud.agent.hdfs.HdfsAgent;
import com.oracle.integration.cloud.agent.hdfs.HdfsAgentFactory;
import com.oracle.integration.cloud.agent.hdfs.HdfsAgentProperties;

public class CreateHdfsAgent {

    public static void main(String[] args) throws Exception {
        // Create the HDFS agent properties
        HdfsAgentProperties properties = new HdfsAgentProperties();
        properties.setHdfsUrl("hdfs://localhost:9000");
        properties.setHdfsUsername("user");
        properties.setHdfsPassword("password");

        // Create the HDFS agent
        HdfsAgent agent = HdfsAgentFactory.create(properties);

        // Start the HDFS agent
        agent.start();

        // Do something with the HDFS agent
        // ...

        // Stop the HDFS agent
        agent.stop();
    }
}

 


Monday, May 1, 2023

Developing a custom connector in Oracle Integration Cloud

 Developing a custom connector in Oracle Integration Cloud involves the following steps:

  1. Define the connector metadata: Start by defining the metadata for the connector. This metadata describes the connector's name, icon, description, and endpoint URL.

  2. Define the connector operations: After defining the metadata, define the connector's operations. These operations describe the actions that the connector can perform. For example, if the connector is for a CRM system, the operations could include creating a new contact, updating an existing contact, or deleting a contact.

  3. Generate the connector SDK: Once the connector metadata and operations have been defined, generate the connector SDK. The SDK provides the code and tools necessary to build and deploy the connector.

  4. Implement the connector functionality: Using the SDK, implement the connector functionality. This involves writing the code that communicates with the target system and performs the operations defined in step 2.

  5. Test the connector: After implementing the connector functionality, test the connector to ensure that it works as expected.

  6. Deploy the connector: Once the connector has been tested, deploy it to Oracle Integration Cloud.

  7. Share the connector: Finally, share the connector with other users in your organization or community.


Create an adapter project

To create an adapter project, you will need to use the Oracle Integration Cloud Adapter Development Kit (ADK). The ADK is a collection of tools and documentation that can help you to develop custom adapters for Oracle Integration Cloud.

To download the ADK, go to the Oracle Integration Cloud website and click on the "Downloads" link.

Once you have downloaded the ADK, extract the contents to a directory of your choice.

Implement the required interfaces

To develop a custom connector, you will need to implement the following interfaces:

  • com.oracle.integration.cloud.adapter.api.IAdapter: This interface provides the basic functionality for an adapter.
  • com.oracle.integration.cloud.adapter.api.IOperation: This interface provides the functionality for a specific operation, such as a query or a save.

Compile and deploy the adapter

Once you have implemented the required interfaces, you can compile and deploy the adapter.

To compile the adapter, use the following command:

javac -d build *.java


To deploy the adapter, use the following command:


java -jar oic-adapter-deployer.jar build


Register the adapter with Oracle Integration Cloud

Once you have deployed the adapter, you can register it with Oracle Integration Cloud.

To register the adapter, go to the Oracle Integration Cloud web console and click on the "Connectors" link.

Click on the "Add Connector" button and select the "Custom Connector" option.

Enter the name and description of the adapter and click on the "Next" button.

Select the adapter JAR file and click on the "Next" button.

Click on the "Save" button to register the adapter.

Once the adapter is registered, you can use it to integrate with other applications.

Thursday, April 27, 2023

Github Actions to Deploy Microservices to Kubernetes on AWS EKS and Oracle Cloud Infrastructure

 Github Actions to Deploy Microservices to Kubernetes

GitHub Actions is a continuous integration and continuous delivery (CI/CD) platform that allows you to automate your build, test, and deployment pipeline. You can create workflows that build and test every pull request to your repository, or deploy merged pull requests to production.

GitHub Actions goes beyond just DevOps and lets you run workflows when other events happen in your repository. For example, you can run a workflow to automatically add the appropriate labels whenever someone creates a new issue in your repository.

Code snippet

name: Deploy Spring Boot to Kubernetes

 

on:

  push:

    branches:

      - master

 

jobs:

  build:

    runs-on: ubuntu-latest

    steps:

      - uses: actions/checkout@v2

      - name: Set up Java

        uses: actions/setup-java@v1

        with:

          java-version: 11

      - name: Build with Maven

        run: mvn clean install

 

  deploy:

    needs: build

    runs-on: ubuntu-latest

    steps:

      - uses: actions/checkout@v2

      - name: Set up Kubernetes

        uses: actions/setup-kubernetes@v1

        with:

          kubeconfig: ${{ secrets.KUBECONFIG }}

      - name: Deploy to Kubernetes

        run: kubectl apply -f deployment.yaml

 

This pipeline will first checkout the code from the repository, then set up Java and build the application with Maven. Once the application is built, it will deploy it to Kubernetes using the kubectl apply command.

Here is a breakdown of each step in the pipeline:

  • on: push: branches: master - This event listener will trigger the pipeline when a new commit is pushed to the master branch.
  • jobs: build - This job will build the Spring Boot application.
  • steps: - These steps are executed in order to build the application.
    • uses: actions/checkout@v2 - This step checks out the code from the repository.
    • name: Set up Java - This step sets up Java on the runner.
    • run: mvn clean install - This step builds the application with Maven.
  • jobs: deploy - This job will deploy the Spring Boot application to Kubernetes.
  • needs: build - This job depends on the build job. This means that the deploy job will not start until the build job has completed successfully.
  • steps: - These steps are executed in order to deploy the application to Kubernetes.
    • uses: actions/checkout@v2 - This step checks out the code from the repository.
    • name: Set up Kubernetes - This step sets up Kubernetes on the runner.
    • with: kubeconfig: ${{ secrets.KUBECONFIG }} - This step sets the Kubernetes configuration file as a secret.
    • name: Deploy to Kubernetes - This step deploys the application to Kubernetes using the kubectl apply command.

Deploy Microservice in Oracle Cloud Infrastructure

 

Deploy Microservice in Oracle Cloud Infrastructure

1.   Create a Compute Instance

First, you need to create a Compute Instance in Oracle Cloud Infrastructure. This is a virtual machine that will host your Spring Boot application.

To create a Compute Instance, follow these steps:

1.   Go to the Compute Instances page in the Oracle Cloud Infrastructure Console.

2.   Click Create Instance.

3.   In the Create Instance dialog, select the Operating System and Image for your Compute Instance.

4.   Click Create.

It will take a few minutes for your Compute Instance to be created.

2.   Install Java and Maven

Once your Compute Instance is created, you need to install Java and Maven. Java is the programming language that Spring Boot is written in, and Maven is a build tool that is used to build and deploy Spring Boot applications.

To install Java and Maven, follow these steps:

1.   SSH into your Compute Instance.

2.   Run the following command to install Java:

Code snippet

sudo apt-get install openjdk-11-jdk

 

3.   Run the following command to install Maven:

Code snippet

sudo apt-get install maven

 

3.   Create a Spring Boot Application

Now that you have Java and Maven installed, you can create a Spring Boot application.

To create a Spring Boot application, follow these steps:

1.   Create a new directory for your Spring Boot application.

2.   In the directory, run the following command to create a new Spring Boot project:

mvn archetype:generate -DgroupId=com.example -DartifactId=my-spring-boot-application -DarchetypeArtifactId=maven-archetype-quickstart

This will create a new Spring Boot project with the name my-spring-boot-application.

4.   Build and Deploy the Application

Once you have created a Spring Boot application, you can build and deploy it.

To build and deploy the application, follow these steps:

1.   In the directory of your Spring Boot application, run the following command to build the application:

mvn clean install

 



2.   Run the following command to deploy the application to Oracle Cloud Infrastructure:

cf push my-spring-boot-application

 

 

This will deploy your Spring Boot application to Oracle Cloud Infrastructure.

5.   Access the Application

Once your Spring Boot application is deployed, you can access it by going to the following URL:

https://my-spring-boot-application.[region].oci.oraclecloud.com

 

 

For example, if your application is deployed in the us-central1 region, the URL would be:

https://my-spring-boot-application.us-central1.oci.oraclecloud.com

 

 

You can now access your Spring Boot application in the Oracle Cloud Infrastructure Console.

 

How IdP Groups Are Tied to Databricks Groups (Unity Catalog)

  🔗 How IdP Groups Are Tied to Databricks Groups (Unity Catalog) 🔑 Key Principle (Read This First) Databricks does NOT “map” IdP groups...