Blog Archives

Tungsten Replicator setting up and Gotchas

Tungsten Replicator site Clicky

So we have a big MYSQL server with mission critical data on it.

I want to replicate it accross to another MYSQL database, they are both ubuntu servers with Percona MYSQL database.

1. Preparation

Master – Install RUBY, JAVA,

Slave – Install RUBY, JAVA

edit the hosts files on both machines in etc/hosts and add the other servers in each one and reference the IP address – just makes things easier.

2. SSH

Make sure you set up SSH so that you can ssh from one server to the other WITHOUT using passwords.

3. PORTS

make sure all ports required 10000,10001 are available (particularly problematical PLESK, WEBMIN a whole host of things like these ports) on all machines. One server actually had a PLESK tool on it even though plesk was not installed!!

4. MYSQL

make sure everything is set up and restore a backup of the database before you start.

5. Helper scripts

Set up your tungsten. You’ll need to edit NODES_MASTER_SLAVE.sh to make sure your nodes are correct and USER_CONFIG as per the tungsten documentation.

install.sh – put in the tungsten folder e.g. tungsten-replicator-2.1.0-343

VERBOSE=1 ./cookbook/install_master_slave ./cookbook/NODES_MASTER_SLAVE.sh

tungsten.sh (actually I renamed this to tungsten and did a chmod +x tungsten)

if [ "$1" == "help" ] || [ "$1" == "" ]; then
 echo "Commands"
 ls installs/cookbook/tungsten/tungsten-replicator/bin/
else
installs/cookbook/tungsten/tungsten-replicator/bin/$1 $2 $3 $4 $5
fi

So you can type

./tungsten replicator status

To run commands.

EJB downloading an excel file from JSP

The Calling Page

Let’s say you have a button on a JSP page that calls it. One of the problems is stopping the page opening as blank. So here’s the calling button

<form method="post" action="#">
 <input type="button" value="Download" onClick="return downloadButton();"/>
        </form>
        

The iFRame is used to trigger the excel call, and here’s the download button javascript.

function downloadButton() {
 
    document.getElementById("excel").src="exportxl.jsp?sql=select * from mytable";
    return false;
    
}

The Responding Page

Now the Calling JSP Page that goes off and gets the Excel Spreadsheet from the EJB, I have an EJB called server and the corresponding remote interface “server remote” note that these must be registered in the xml files in the web app so the web app can see the remote interface in the EJB

<%-- 
    Document   : exportxl
    Created on : Aug 1, 2011, 8:36:23 AM
    Author     : paulc
--%>
<%@ page import = "java.sql.*" %>
<%@ page import = "javax.naming.Context" %>
<%@ page import = "javax.naming.InitialContext; "%>
<%@ page import = "PHJ.*" %>
<%@page contentType="application/vnd.ms-excel" pageEncoding="UTF-8"%>
<%

    PHJ.ServerRemote jobManagerBean = null;
    try {
        Context c = new InitialContext();
        jobManagerBean = (PHJ.ServerRemote) c.lookup("java:comp/env/Server");
    } catch (Exception ne) {
        System.out.println("Could not find Job Manager Bean " + ne);
        out.println("Could not find Job Manager Bean");
        return;
    }

    response.setHeader("Content-disposition", "attachment; filename=excelexport.csv");

    String sql = "";
    if (request.getParameter("sql") != null) {
        sql = request.getParameter("sql");

        out.print(jobManagerBean.prepareCSVForExcel(sql));

    } else {
        out.println("Error, No SQL statement specified");

    }
    
    

%>

The EJB

And here’s the routine in the EJB.

 
    private Connection conn;
    private String dbName = "myJNDIRESOURCE"; // make sure you have JNDI resource setup 
    /* This routine gets a connection from the above DBName, however, if it can't find it it lists the available JNDI resourcees
     * to the console or log
     */ 
    private Connection getConnection() {
        if (conn != null) {
            try {
                if (!conn.isClosed()) {
                    return conn;
                }
            } catch (SQLException ex) {
                // Do nothing just carry on.
            }

        }
        InitialContext ic = null;

        Query q = null;

        try {
            // initialize JNDI lookup parameters
            ic = new InitialContext();
            DataSource ds = (DataSource) ic.lookup(dbName);
            conn = ds.getConnection();
        } catch (Exception e) {
            try {
                System.out.println("JNDI Resource " + dbName + " not found " + e);
                NamingEnumeration children = ic.list("");
                while (children.hasMore()) {
                    NameClassPair ncPair = (NameClassPair) children.next();
                    System.out.print(ncPair.getName() + " (type " + ncPair.getClassName() + ")");

                }
                System.out.println("**** CANNOT FIND DATABASE ************************");
                children = ic.list("jdbc");
                while (children.hasMore()) {
                    NameClassPair ncPair = (NameClassPair) children.next();
                    System.out.print(ncPair.getName() + " (type " + ncPair.getClassName() + ")");

                }
                System.out.println("****************************************");
            } catch (Exception f) {
                System.out.println("Error 57:" + f);
            }
            System.out.println("ERROR 52: " + e);
            return null;
        }
        return conn;

    }

    public String prepareCSVForExcel(String sSQL) {

        String sResult = "";
        try {
            conn = getConnection();
            Statement s = conn.createStatement();
            ResultSet r = s.executeQuery(sSQL);
            // feldnames
            for (int i = 1; i != r.getMetaData().getColumnCount(); ++i) {
                sResult = sResult + r.getMetaData().getColumnName(i) + ",";


            }
            sResult = sResult + "\n";

            while (r.next()) {
                for (int i = 1; i != r.getMetaData().getColumnCount(); ++i) {
                    try {
                    if (r.getObject(i) != null) {
                        sResult = sResult + r.getString(i) + ",";
                    } else {
                        sResult = sResult + ",";

                    }
                    } catch (Exception e) {
                        sResult = sResult + ","; // Happens when there is a conversion error to a tiemstamp
                    }


                }
                sResult=sResult+"\n";



            }




        } catch (Exception e) {
            return e + "";
        }
        return sResult;
    }

Android Global Variables.

Those of you familiar with android programming will be used to using intents to pass data from activity to activity but there is a much easier way of storing state information. Using a global variable class.

1. Create a class

public class GlobalVariables extends Application {

    public String sResult = "";
}

This extends the application class.

Then edit your AndroidManifest.xml file

   <application android:name="GlobalVariables" android:theme="@style/CustomTheme" android:label="@string/app_name"
                 android:icon="@drawable/icon"  >

When you want access to one of those juicy globals in your application just use:-

  GlobalVariables gb = ((GlobalVariables) getApplicationContext());
  Log.i("Test Variable is "+gb.sResult);

Simples!

 

I think I’m in love….and not just with my wife

We’ve got a new member of our programming team, Uthay, I can’t even say or spell his full name, but anyway, he came along and showed me this tool called SQLyog.

Although we only had the community edition after some coaxing we managed to get the boss to shell out for the full version, enterprise edition.

This is absolutely superb, it is WELL WORTH upgrading the full enterprise version has some amazing tools that make database development MUCH easier. Especially when like us we have to modify and change databases then roll out the changes to another server. For example when we have 5 or 6 versions of our application out there and want to roll out the new application, that’s easy enough, but remembering the differences between the two versions is a bit of a pain, with this tool we can just syncronize them and it will modify the destination tables (and data if we want) to match the source.

I particularly like the Query development tool, one of the good things about good old Microsoft Access was how easy it was to build complex queries, in this software it is just as easy, particularly useful when building large scale data structures.

It’s such a shame it doesn’t support anything else apart from MYSQL. However I will be transmographying a database from a SQL server over to MYSQL JUST TO USE IT!

http://www.webyog.com/en/

There is a community version but the full version is only like £50 so it’s well worth purchasing.

MySQL is Not pronounced MY SQL

Ok – this is not exactly hot news. But I went to an EXCELLENT seminar on Friday last week with Sun and MYSQL. Which according to the founder is pronounced “MESQL” as the “MY” is the name of his daughter. Well, translated it’s “MY” not “ME”, I tried saying MESQL but it makes me sound like a Texan. (sorry if you Texan). So I’m sticking to MYSQL.

They spent a lot of time trying to convince us that the open source software model (that Sun use) is the way to go for software. You know, install it, make it work – if you want support pay for it. I liked this for a number of reasons and didn’t like it for other reasons.

Why I Like Open Source

– Customer and you can get the software, run it, test it, see if it’s what they need and then upgrade to a commercial license and support whent they are happy
– Customer can get the “best of breed” and mix and match
– Source code is available if supplier goes bust (ok, I know Sun & MYSQL are not going bust but it’s a thought)
– For the developer its less pressure than working on a project where you are up against deadlines for a client. However this also means the development may be less focussed on solving a particular problem, one of the comments was that Open Source payroll “Isn’t mature enough yet”, well that’s fine but its not cheese. It won’t just mature, someone, somewhere has to fund the development (see bad points). In this model:- 

a. Customer pays a license fee to use the commercial version (they call it Enterprise in the Open Source World)
b. Customer pays a support fee to use the commercial version – or hires in house people to support it
c. Customer pays when they are happy or at least ready to get professional help.
d. I like the 15 minute rule, if I can’t install it and have it running in 15 minutes (even though it mike take 2 hours) to download then its probably not ready.
– Open source is better for the smaller company because they can get other developers to contribute and people to file bug reports for example when something is not working.
– Open source means that developers get recognition in the developer community for work they have done especially if it is pioneering. Developers need encouragement and sometimes in a small team they don’t get it. Especially being the only developer in a company, it’s unlikely that those outside of the software engineering world have any understanding about the elegance and effort put into a peice of software, I’ve seen bad software that looks great but is terrible and great software that looks terrible but works like a dream. (often us developers need to hire a designer!).

Bad Points about open source

– Developers need to feed their kids – so unless someone funds the development then Developer doesn’t feed kids until someone pays a support contract, most developers are not rich enough to live on thin air.
– Unless someone funds it a project may take longer to develop than with the conventional hire a developer model and not be “quite” what the customer was looking for in the first instance. Thus the best open source software is the GENERAL stuff, that is not focused on doing a specific job, like payroll or transport. Because from the base platform you can build up to do something specific.

So, it’s scarey and nice at the same time. Yes, I love open source but Yes, I need to be paid to develop so I can feed my family (and they are hungry, especially the cat).

So what about MySQL itself, well it’s great, thats all there is too it. I love the way I can change database engines PER TABLE which means that for instance if I have an archive I can just use an archive database, I love the clustering, I even like the logo. Good job.

Sun buying it? Well I wrote an article about this on this blog anyway so you can trawl through if you like and read it.

Great Seminar, well worth going, only thing was, not enough freebies. Good grief Sun I travelled an hour accross London. I expect at least a CD and T-Shirt, not just a pen! A PEN!, ok they were giving away an IPhone, which was nice but I never found out who got that. I guess one of the big banks who were there.

If you fancy doing the presentations to yourself you can download them here.

Solaris 10

Right

1. Take one DVD copy of Solaris 10
2. Find a machine it will mostly run on – older machines tend not to work (use UBUNTU, that runs on any old dog).
3. Install
4. Install WEBMIN (PLEASE save yourself any grief )
5. Use webmin (http://localhost:10000) to set up your users (use root and your root password to login)
6. Login as the new user

Now you’re ready to work

Running Virtual Box

I couldn’t for the life of me be bothered to faff around with getting Virtual Box running. If Sun own it, it should just install and go on a Sun operating system. So instead after some whining at the purchasing manager here I got a copy of win4solaris which runs FAULTLESSLY, its fast, looks amazing and allows me to work on the sad Microsoft Access projects I have to deal with.

Running everything else

Well I’ve been using Solaris for about a week now so here goes with the things to cheer about and the things to complain about!

Cheers! Hooray!

It’s very very stable, very very very stable. It runs as fast when you turn it on as it does when you’ve been running for hours. Even with Second Life, Netbeans and other things running.

This makes my life a joy because I’m not constantly crashing or having problems.

Its FREE.

It runs Second Life (very important).

Runs Netbeans beautifully.

Has great desktop backgrounds (little things please little minds!).

Networking speed – it ROCKS when the windows users are complaining its going slow again mine is running like the proverbial wind. (perhaps greedy Solaris is stealing all the bandwidth from the Windows users).

Sad! Boohoo!

Come on Sun! Install a CONTROL PANEL. Somewhere to manage users, groups, etc like Windows has! The problem with Solaris is that you have to be willing to get inside the box and dig to use it. This could be (like Ubuntu which I think is awesome) a first rate operating system. Ok I can install webmin, fine – so bundle webmin and have a link to it from the desktop.

Also it should have something like the “APT-GET” utility in Ubuntu. (I’m not linking it again ok, just use the link above). Like a software search facility gui which also installs the packages… ok ok I’ll write somethig when I get time.

There’s a couple of projects for me when I get some spare time! (IF I ever get any spare time).

Netbeans 6.1 Beta is announced

Well today I’ve loaded up Netbeans 6.1, fantastic  as usual, it’s no wonder they one that award!

So Netbeans 6.1 is here. It’s much faster than the last one at loading, which I have to say is slightly annoying as it used to take just under the time it took to make a cup of tea. Now I can barely get the milk from the fridge. Ah well.

I’ve not actually had time to dig into some of the new features but the speed of it is fantastic. I’m too busy building 4, yes 4 enterprise apps in it. In a few days I’ll write a more complete review but it seems very stable and works like a dream.

Just one thing. I’m getting bored of not coding anything substantial because the web apps are just being laid out. I am one of those programmers who likes to be challenged not given something that is too easy. They are words I never thought I’d use let alone think about anything to do with Java.

Connection to two MYSQL databases through glassfish at once.

This one had me foxed for a bit, I had to connect to two different MYSQL databases in the same app.

The problem was that the MYSQL driver that I had installed with Glassfish was an old one. As soon as I updated it and then changed the database links in Netbeans (so they would roll out to Glassfish) and then it works like a dream, in fact I noticed that the latest MYSQL driver is considerably faster being that I have a remote database coming into a netbean and out to the client.

On the topic of Netbeans, they are WONDERFUL. I love writing code in beans and having it pull the data, sort it out and send it to whatever-client-I-so-decide whether web based, app based or mobile (or something else).

Lastly I’ve installed the latest Woodstock web componants into NB for a new project I’m doing and they are really great.

Sun Microsystems and MYSQL get Married

Wedding bells are in the air. But what does this mean?

The marriage of Sun Microsystems  and MYSQL means quite a lot. It is a very strategic move that Mr Msoft should have thought of. With Netbeans fast becoming a favorite amongst developers it means MYSQL could become the defacto back end database standard, leaving Mr Msoft sadly not invited to the reception.

For a developer learning Netbeans from .net is not rocket science, neither is working from Mr Msofts database server over to sunny MYSQL. So we could see a new breed of developer who’s first choice is always the MYSQL/Netbeans platform for anything from a small website up to a full enterprise system with glassfish.

Perhaps it could even become the point that servers will be rolled out with Ubuntu Linux on since we don’t really care what we run our java/mysql on – it makes no difference.

One of the main objections to java as a language and development tool is the cost of ownership is higher in the long run because you have to have better programmers, oh, sorry, did I say better, I meant more expensive, so you’re initial rejoicing at getting a cheap start up base becomes dogged by the fact you’re spending $100 US a day more on programmers to roll out the “Widget Stock System”. (man I’m biased).

Times are changing and I think for the better. Netbeans is a true RAD tool and now that MYSQL is part of the family you can build an enterprise system with Mr Msoft even getting a look in, except perhaps on the client machines operating system unless the user is running Linux or Mac.

So every blessing to your new marriage and I hope the honeymoon lasts forever. The nice thing is – we are all invited, at no cost.