[spunk]

Hi everyone:

I have this problem, I'm currently developing a custom made software and I'm just starting and this came up and I've searching all over and I haven't found an answer. Let me try to break it down as simple as possible:

I have three computers that are connected to mechanical machines that perform quality testing on different types of foam. After those tests are done, the software that its running on each computer collects data, raw data saved in text files. Each machine makes a different test to the same foam.

My problem:

I have to somehow collect the raw data and store that data in a database in a single step, meaning: i can't use a batch file to store that same data. As soon as the software retrieves the data, it should store it in a database.

Why in the java forum? Cause I believe it can be done using a servlet.

I hope I was able to explain myself. If anyone can help me, I'd be thankful.

Thank you again guys for taking your time to read this

Hugo

[stdunbar]

I'm sure you could implement this as a servlet or, perhaps, as a web service. The web service route would allow you to have a single point of data insertion so that the data collection machines would not directly interface with the db.

Let us know some more information to help you move forward.

[King Coder]

Can you explain the situation and architecture a little more? Are these computers connected to the internet and from there you need to connect to DB?

[spunk]

It is all done as an intranet. There is (still) no connection to the outside (internet). It will probably happen in the future, but that's not what is going to happen right now.

Check here for the representation i've done so far. On the left there are represented three machines and corresponding computers. Those machines send data to the computers through USB cables and a software running on the computer saves that data in text files (or not, it only saves if you hit 'Export'). The thing is, the same kind of product is being tested in three different machines and all data is separated from eachother logically and physically. The idea is to store all of that data under a unique ID to be consulted later on.

Fortunatly for me, there are some good news. The programs source is available and I'll probably be authorized to modify them.

It is not a case of data migration. Its about the storage of data each time the program receives data that comes from the testing machine.

Thank you both for your replies.

[stdunbar]

How do the various monitors communicate normally - just text files? If you didn't want to modify the monitor code you could have a process on each of the monitor machines that looks for a new text file. That could pick it up and send it directly to a servlet. That could use an HTTP POST or web services.

As you point out each monitor would need some sort of unique identifier. This could be something as simple as the IP address of the machine but may need to be something more depending on your network.

One thing I'm a bit confused about in your picture - what are the stored images? Do the monitor machines generate images of some sort? Do you mean images like jpg's or images like some sort of binary image? Either way if you are transferring binary data from the monitors to the servlet you'll need to do a bit more work by sending a multipart form (normal HTTP and HTML) or a soap attachment (web services). It really isn't too bad but it does add to the work a bit.

[spunk]

Each monitor exports text files and stores them in a local folder. I've thought about having a process that could detect when a text file has been written and then parse that data and send it to the database. But it's not flawless. If by accident someone copied a text file to that folder the process would just try to write that file in the database and then everything would go wrong. And if something can go wrong, it will go wrong.

The unique identifier would be the formula number that its beeing tested. That problem is solved. They never had an IT guy here before, so their network is not really solid and IP's are atributed dinamically.

One of the machines is a scanner and the output is of course, an image. A local software will read that image and will output text data. But the image itself its lost. Apart from storing data it should also store images. That image should be store for later use (reporting and such). Images are exported in jpg.

I'll probably just use XML as an exporting measure instead of HTML and POST methods. Shouldnt be as painful as you so politely put it. Just kidding.

And let me tell you guys.. designing the DB for this company will be hell :S