Showing posts with label Cluster. Show all posts
Showing posts with label Cluster. Show all posts

Setting Up, Using and troubleshooting a Weblogic 12c Cluster

In this post reader can easily understand the weblogic clustering concept. Topics Covered in this post are :

  1. What is a WebLogic Cluster?
  2. Set up & configuration
  3. Best practices
  4. Troubleshooting
  5. Dynamic Clustering (WLS 12.1.2 and up)
  6. Cluster Targeted JMS Servers (WLS 12.1.2 and up)

1. What is a WebLogic Cluster?

Multiple server instances running simultaneously and working together to transparently provide clients with increased scalability and reliability for servers, WebApp, WS, EJB apps, JDBC resources, JMS and more A cluster appears to clients to be a single server instance All clustered servers must run the same WLS version and belong to the same domain

Benefits of clustering

Failover : Objects copied to a secondary server that will finish the task in case the primary server is unavailable for any reason
Migration:  Relocation of a full server in the event of a failure
Load balancing :  Uninterrupted availability and even distribution of tasks  among instances

Load Balancing for HTTP clusters

Proxy Servers can be used to provide load balancing (round-robin by default) and failover for a cluster . Example: OHS, IIS, iPlanet

External Hardware LB .  Example: F5 Networks Big-IP

Basic Cluster Proxy Architecture

Basic Cluster Proxy Architecture

2. Setup and Configuration

Domain Configuration wizard

Domain Configuration wizard

assign server to cluster

WebLogic Administration Console

WebLogic Administration Console

WebLogic Administration Console

Weblogic Scripting Tool (WLST)

connect('weblogic','weblogic', 't3://localhost:7001')


  • Architecture
        Basic or multi-tier architecture :  The recommended multi-tier architecture uses one cluster to serve static HTTP content and clustered servlets, and one cluster to serve clustered EJBs

  • Proxy Plug-Ins or physical LB  
           – HttpClusterServlet, iPlanet, OHS or IIS

  • Messaging Mode
             Multicast or Unicast, to broadcast availability of services and heartbeats  IP sockets for peer-to-peer communications between instances
  • Replication Groups
             Preferred list of clustered instances to be used for storing session state  replicas
  • HTTP Sessions replication
              In-memory, file-based, JDBC-based persistence, Coherence*Web

3. Clustering Best Practices



  • Admin server should not be part of a cluster  
            -->  Main purpose of this server is to perform the bulk of the processing for the domain
            -->  Administrative objects are not clusterable
  •  Http Session States need to persist in memory to enable automatic failover of servlets and JSPs
  • Deploy clusterable objects to the cluster, rather than to individual Managed Servers in the cluster

HTTP State Management Best Practices

  •  Create WLS machines if you are replicating the state across servers on different physical machines
  • Use replication groups to define the failover strategy
  • Choose the most appropriate replication strategy depending on the application needs and architecture
  • Use the ServerDebugConfig MBean to track session replication problems
  • Ensure that objects placed in replicated sessions are serializable

Managing HTTP Sessions in a Cluster

Multicast vs Unicast

  • Unicast is better suited for small clusters (less than 20 instances)
  • With over 50 members, ensure that the network allows multicast
  • With 100 members, try to break up the domain into multiple separate domains and individual clusters for better administration and management even though Multicast option can handle such large domains

WebLogic Server Cluster Messaging Protocols

DataSource on Clusters

  • Targeting a data source to a cluster will not enable failover of connections - it will ease the process of reconnecting when a connection fails
  • Multi Data Source to guaranty data source failover  
  • JDBC is a highly stateful client-DBMS protocol, in which the DBMS connection and transactional state are tied directly to the socket between the DBMS process and the client (driver). For this reason, failover of a connection while it is in use is not supported


Troubleshooting Replication Issues

  • Enabling Replication Debugs to Track Session Replication Failures, often overwhelming and filing up logs rapidly with complex data
  • Set the flags from the command line during server startup by adding the following options:
-Dweblogic.debug.<DebugName>=true in addition to

– DebugClusterAnnouncements
  • Debug each Announcement, StateDump, and Attributes msg sent or received by multicast
– DebugFailOver
  • Debug stub -level fail-over processing
– DebugReplication
  • Debug high-level cluster replication information
– DebugReplicationDetails
  • Debug low-level cluster replication information
– DebugCluster

Troubleshooting Multicast Issues

  • utils.MulticastTest
  • Set these flags from the command line during server startup by adding the following options:

HTTP Load Balancing Problems in Cluster Using Proxy Plug-in

  • Configure the proxy plug-in to produce additional diagnostic data
– Enable proxy debug by setting Debug="ALL" in the proxy configuration file
– Enable proxy bridge by setting DebugConfigInfo="ON" in the proxy configuration file
  • Analyze the Proxy Log File and the Proxy Bridge Page
– http://webserver_host:port/path/xyz.jsp?__WebLogicBridgeConfig

  • Search for X-WebLogic-Cluster-List in proxy log to find when a truncated list has been returned from backend cluster node. Determine which backend cluster node has returned the truncated X-WebLogic-Cluster-List by following the preceding log entries for the same request
  • Look for "Exception objects created" under "Runtime statistics" on the proxy bridge page. If the count is greater than 0, then exceptions have occurred (logged in the proxy log)

    5. Dynamic Clustering (WLS 12.1.2 and up)

    • Introduced feature in WebLogic Server 12.1.2
    • Scale up dynamically the number of server instances in your domain
    • A dynamic cluster is any cluster that contains one or more dynamic servers

    Server template

    • Dynamic clusters are based on a single shared server template used to specify the configuration of dynamic servers
    • A dynamic cluster can be created via WLST 

    Creating Dynamic Clusters

    • Specify how many servers are needed at peak load and define a template to use as the base configuration those servers 
    • When additional server capacity is needed, you can then start a server instance without having to first manually configure it and add it to the cluster


    • Individual targeting is not supported
    • Server-specific configuration attributes (e.g. replication groups, user preferred server, hosting server etc)

    6. Cluster Targeted JMS Servers (WLS 12.1.2 and up)

    JMS Server and Cluster before 12.1.2

    • To leverage clustering capabilities of Weblogic JMS such as load balancing and high availability, you need to configure Distributed Destinations
    • Connection factories (resources that enable JMS clients to create JMS connections) can be deployed on multiple WebLogic Server instances
    • Connection factories are bound to the JNDI tree for multiple server instances

    Cluster Targeted JMS Servers

    • Targeting JMS Servers and optionally persistent stores directly to a cluster
    • Enable dynamic scaling of JMS resources in a dynamic cluster or the dynamic members of a mixed cluster (individually configured servers along with dynamic servers)


    • Consider known limitations before developing  applications using dynamic clusters and cluster targeted JMS Servers, example:
                   --No support for Automatic Server Migration (ASM)
                   --Store-and-Forward agents cannot be targeted to a dynamic or mixed server
                   --No support for Replicated distributed topics (RDTs) when any member destination is                           hosted on a cluster targeted JMS server