SQL Database Persistence

[Michael MacFadden]

All,

I was curious if there was any interest in a persistence module that
would store accounts, deltas etc in a SQL database (mysql, postgresql,
etc). Thanks.

~Michael

[James Purser]

Sure,

Something like a mysql module would be excellent as it would allow people to
tie into existing infra (mysql has a massive install base).

James

On Wed, Dec 1, 2010 at 9:15 AM, Michael MacFadden <

Post by Michael MacFadden
All,
I was curious if there was any interest in a persistence module that
would store accounts, deltas etc in a SQL database (mysql, postgresql,
etc). Thanks.
~Michael
--
You received this message because you are subscribed to the Google Groups
"Wave Protocol" group.
To unsubscribe from this group, send email to
.
For more options, visit this group at
http://groups.google.com/group/wave-protocol?hl=en.

[James Purser]

And one other thing. We need delta persistance in some form or another.

What's the status on the various persistance modules?

James

Post by James Purser
Sure,
Something like a mysql module would be excellent as it would allow people
to tie into existing infra (mysql has a massive install base).
James
On Wed, Dec 1, 2010 at 9:15 AM, Michael MacFadden <

Post by Michael MacFadden
All,
I was curious if there was any interest in a persistence module that
would store accounts, deltas etc in a SQL database (mysql, postgresql,
etc). Thanks.
~Michael
--
You received this message because you are subscribed to the Google Groups
"Wave Protocol" group.
To unsubscribe from this group, send email to
.
For more options, visit this group at
http://groups.google.com/group/wave-protocol?hl=en.

[Michael MacFadden]

I was thinking of implementing this via the Java Persistence API, such
that it would be (in theory) database agnostic. At the very least I
would try to support mysql, postgresql, and maybe derby (for a simple
filesystem based database).

I am not sure what the status on the other modules is, but what I was
thinking is that a SQL persistence module is nice if you plan on ever
being able to cluster the Wave in a Box implementation. Purely file
based implementations can become problematic if you are clustering or
load balancing across multiple machines. Of course this is a bit of a
hypothetical anyway since the current server isn't really cluster-able
yet.

~Michael

Post by James Purser
And one other thing. We need delta persistance in some form or another.
What's the status on the various persistance modules?
James

Post by James Purser
Sure,
Something like a mysql module would be excellent as it would allow people
to tie into existing infra (mysql has a massive install base).
James
On Wed, Dec 1, 2010 at 9:15 AM, Michael MacFadden <

Post by Michael MacFadden
All,
I was curious if there was any interest in a persistence module that
would store accounts, deltas etc in a SQL database (mysql, postgresql,
etc).  Thanks.
~Michael
--
You received this message because you are subscribed to the Google Groups
"Wave Protocol" group.
To unsubscribe from this group, send email to
.
For more options, visit this group at
http://groups.google.com/group/wave-protocol?hl=en.

[Patrick Nagel]

Hi,

Post by Michael MacFadden
I was curious if there was any interest in a persistence module that
would store accounts, deltas etc in a SQL database (mysql, postgresql,
etc). Thanks.

Yes, SQL-based persistence is certainly of interest.

Patrick.

[James Purser]

Well setting up the basics (account management shouldn't be too hard).

It's the delta storage thats going to cause the main issue I think.

James

Post by Patrick Nagel
Hi,

Post by Michael MacFadden
I was curious if there was any interest in a persistence module that
would store accounts, deltas etc in a SQL database (mysql, postgresql,
etc). Thanks.

Yes, SQL-based persistence is certainly of interest.
Patrick.
--
Key ID: 0x86E346D4 http://patrick-nagel.net/key.asc
Fingerprint: 7745 E1BE FA8B FBAD 76AB 2BFC C981 E686 86E3 46D4

[Chris Harvey]

FYI, the iotaWave server is built purely on a MySQL database, accessed via
JDBC from Java. This has allowed us to tightly integrate the wave-related
tables (blips, waves, wavelets, threads, deltas, etc.) with all of the other
eMail and business functionality that we support.

The high-performance requirements of Wave operation requires some thought as
to database schema design, especially indexing.
--
Chris
iotawave.org
Singapore

[Michael MacFadden]

Thanks for your thoughts. I am not sure exactly when I might start
working on this, but I think there is at least enough interest to put
it on the list of things to be done / looked in to at some point.
Thanks.

Post by Chris Harvey
FYI, the iotaWave server is built purely on a MySQL database, accessed via
JDBC from Java. This has allowed us to tightly integrate the wave-related
tables (blips, waves, wavelets, threads, deltas, etc.) with all of the other
eMail and business functionality that we support.
The high-performance requirements of Wave operation requires some thought as
to database schema design, especially indexing.
--
Chris
iotawave.org
Singapore

[Alex North]

That sounds great. I'd very much like to see an SQL-backed storage
implementation. I have some experience and ideas about how to do it, so
if/when you get round to designing it perhaps I can help.

Alex

Post by Michael MacFadden
Thanks for your thoughts. I am not sure exactly when I might start
working on this, but I think there is at least enough interest to put
it on the list of things to be done / looked in to at some point.
Thanks.

Post by Chris Harvey
FYI, the iotaWave server is built purely on a MySQL database, accessed

via

Post by Chris Harvey
JDBC from Java. This has allowed us to tightly integrate the wave-related
tables (blips, waves, wavelets, threads, deltas, etc.) with all of the

other

Post by Chris Harvey
eMail and business functionality that we support.
The high-performance requirements of Wave operation requires some thought

as

Post by Chris Harvey
to database schema design, especially indexing.
--
Chris
iotawave.org
Singapore

--
You received this message because you are subscribed to the Google Groups
"Wave Protocol" group.
To unsubscribe from this group, send email to
.
For more options, visit this group at
http://groups.google.com/group/wave-protocol?hl=en.

[Vega]

I just have a question - will SQL DB be an additional requirement to
Mongo DB or an alternative requirement?

Post by Alex North
That sounds great. I'd very much like to see an SQL-backed storage
implementation. I have some experience and ideas about how to do it, so
if/when you get round to designing it perhaps I can help.
Alex

Thanks for your thoughts.  I am not sure exactly when I might start
working on this, but I think there is at least enough interest to put
it on the list of things to be done / looked in to at some point.
Thanks.

Post by Chris Harvey
FYI, the iotaWave server is built purely on a MySQL database, accessed

via

Post by Chris Harvey
JDBC from Java. This has allowed us to tightly integrate the wave-related
tables (blips, waves, wavelets, threads, deltas, etc.) with all of the

other

Post by Chris Harvey
eMail and business functionality that we support.
The high-performance requirements of Wave operation requires some thought

as

Post by Chris Harvey
to database schema design, especially indexing.
--
Chris
iotawave.org
Singapore

--
You received this message because you are subscribed to the Google Groups
"Wave Protocol" group.
To unsubscribe from this group, send email to
.
For more options, visit this group at
http://groups.google.com/group/wave-protocol?hl=en.

[James Purser]

It will be on the same level as mongodb, the file based persistance module
and so on.

James

Post by Vega
I just have a question - will SQL DB be an additional requirement to
Mongo DB or an alternative requirement?

Post by Alex North
That sounds great. I'd very much like to see an SQL-backed storage
implementation. I have some experience and ideas about how to do it, so
if/when you get round to designing it perhaps I can help.
Alex

Post by Michael MacFadden
Thanks for your thoughts. I am not sure exactly when I might start
working on this, but I think there is at least enough interest to put
it on the list of things to be done / looked in to at some point.
Thanks.

Post by Chris Harvey
FYI, the iotaWave server is built purely on a MySQL database,

accessed

Post by Alex North

Post by Michael MacFadden
via

Post by Chris Harvey
JDBC from Java. This has allowed us to tightly integrate the

wave-related

Post by Alex North

Post by Michael MacFadden

Post by Chris Harvey
tables (blips, waves, wavelets, threads, deltas, etc.) with all of

the

Post by Alex North

Post by Michael MacFadden
other

Post by Chris Harvey
eMail and business functionality that we support.
The high-performance requirements of Wave operation requires some

thought

Post by Alex North

Post by Michael MacFadden
as

Post by Chris Harvey
to database schema design, especially indexing.
--
Chris
iotawave.org
Singapore

--
You received this message because you are subscribed to the Google

Groups

Post by Alex North

Post by Michael MacFadden
"Wave Protocol" group.
To unsubscribe from this group, send email to
.
For more options, visit this group at
http://groups.google.com/group/wave-protocol?hl=en.

--
You received this message because you are subscribed to the Google Groups
"Wave Protocol" group.
To unsubscribe from this group, send email to
.
For more options, visit this group at
http://groups.google.com/group/wave-protocol?hl=en.

[Alex North]

Alternative, I would hope.

Post by Vega
I just have a question - will SQL DB be an additional requirement to
Mongo DB or an alternative requirement?

Post by Alex North
That sounds great. I'd very much like to see an SQL-backed storage
implementation. I have some experience and ideas about how to do it, so
if/when you get round to designing it perhaps I can help.
Alex

Post by Michael MacFadden
Thanks for your thoughts. I am not sure exactly when I might start
working on this, but I think there is at least enough interest to put
it on the list of things to be done / looked in to at some point.
Thanks.

Post by Chris Harvey
FYI, the iotaWave server is built purely on a MySQL database,

accessed

Post by Alex North

Post by Michael MacFadden
via

Post by Chris Harvey
JDBC from Java. This has allowed us to tightly integrate the

wave-related

Post by Alex North

Post by Michael MacFadden

Post by Chris Harvey
tables (blips, waves, wavelets, threads, deltas, etc.) with all of

the

Post by Alex North

Post by Michael MacFadden
other

Post by Chris Harvey
eMail and business functionality that we support.
The high-performance requirements of Wave operation requires some

thought

Post by Alex North

Post by Michael MacFadden
as

Post by Chris Harvey
to database schema design, especially indexing.
--
Chris
iotawave.org
Singapore

--
You received this message because you are subscribed to the Google

Groups

Post by Alex North

Post by Michael MacFadden
"Wave Protocol" group.
To unsubscribe from this group, send email to
.
For more options, visit this group at
http://groups.google.com/group/wave-protocol?hl=en.

--
You received this message because you are subscribed to the Google Groups
"Wave Protocol" group.
To unsubscribe from this group, send email to
.
For more options, visit this group at
http://groups.google.com/group/wave-protocol?hl=en.

[Tad Glines]

On Tue, Nov 30, 2010 at 2:15 PM, Michael MacFadden <

Post by Michael MacFadden
All,
I was curious if there was any interest in a persistence module that
would store accounts, deltas etc in a SQL database (mysql, postgresql,
etc). Thanks.
~Michael

I have a lot of interest in this as well. I've attached a data model I
created last year. I tried to balance several factors (normalization, space
etc...) in the model. For example, instead of using the actual
waveid/waveletid as foreign keys, I used a bigint: possibly an unnecessary
optimization/complication.

We may want to consider using JDO <http://db.apache.org/jdo/> (supports JDO
and JPA) and Derby <http://db.apache.org/derby/> as a starting point. Toss
in a cache (e.g. ehcache, etc...) to add scalability.

-Tad

[Tad Glines]

Apparently attachments are not allowed on apache lists.
Here's a link to the
image<https://docs.google.com/drawings/edit?id=12RSnga-D2WtyBUVnSQPG2Vjfk2ttxaLaoTH34y2Y7tc&hl=en&authkey=CKr-_L0N>
.

-Tad

Post by Tad Glines
On Tue, Nov 30, 2010 at 2:15 PM, Michael MacFadden <

Post by Michael MacFadden
All,
I was curious if there was any interest in a persistence module that
would store accounts, deltas etc in a SQL database (mysql, postgresql,
etc). Thanks.
~Michael

I have a lot of interest in this as well. I've attached a data model I
created last year. I tried to balance several factors (normalization, space
etc...) in the model. For example, instead of using the actual
waveid/waveletid as foreign keys, I used a bigint: possibly an unnecessary
optimization/complication.
We may want to consider using JDO <http://db.apache.org/jdo/> (supports
JDO and JPA) and Derby <http://db.apache.org/derby/> as a starting point.
Toss in a cache (e.g. ehcache, etc...) to add scalability.
-Tad

[Asif Khan Gillani]

Hi

as a quick and a simple solution can't we modify the existing File
based implementation to be used with DB's. I mean we can have a single
table to hold custom file objects containing file content and rest of
the process remains same as that in File based implementation.

Asif Khan Gillani

Post by Tad Glines
Apparently attachments are not allowed on apache lists.
Here's a link to the
image<https://docs.google.com/drawings/edit?id=12RSnga-D2WtyBUVnSQPG2Vjfk2t...>
.
-Tad

Post by Tad Glines
On Tue, Nov 30, 2010 at 2:15 PM, Michael MacFadden <

Post by Michael MacFadden
All,
I was curious if there was any interest in a persistence module that
would store accounts, deltas etc in a SQL database (mysql, postgresql,
etc).  Thanks.
~Michael

I have a lot of interest in this as well. I've attached a data model I
created last year. I tried to balance several factors (normalization, space
etc...) in the model. For example, instead of using the actual
waveid/waveletid as foreign keys, I used a bigint: possibly an unnecessary
optimization/complication.
We may want to consider using JDO <http://db.apache.org/jdo/> (supports
JDO and JPA) and Derby <http://db.apache.org/derby/> as a starting point.
Toss in a cache (e.g. ehcache, etc...) to add scalability.
-Tad

[Asif Khan Gillani]

I mean single table for each kind of Store ..

Post by Tad Glines
Apparently attachments are not allowed on apache lists.
Here's a link to the
image<https://docs.google.com/drawings/edit?id=12RSnga-D2WtyBUVnSQPG2Vjfk2t...>
.
-Tad

Post by Tad Glines
On Tue, Nov 30, 2010 at 2:15 PM, Michael MacFadden <

Post by Michael MacFadden
All,
I was curious if there was any interest in a persistence module that
would store accounts, deltas etc in a SQL database (mysql, postgresql,
etc).  Thanks.
~Michael

I have a lot of interest in this as well. I've attached a data model I
created last year. I tried to balance several factors (normalization, space
etc...) in the model. For example, instead of using the actual
waveid/waveletid as foreign keys, I used a bigint: possibly an unnecessary
optimization/complication.
We may want to consider using JDO <http://db.apache.org/jdo/> (supports
JDO and JPA) and Derby <http://db.apache.org/derby/> as a starting point.
Toss in a cache (e.g. ehcache, etc...) to add scalability.
-Tad

[John Blossom]

It certainly would make sense. The LAMP stack will be there by default, and
if people want to use another SQL-serving mechanism they may. Hopefully this
would help to manage these state issues more flexibly, otherwise there's the
probability that the "simple" file solution becomes a bespoke database
solution that will be very troublesome to maintain.

[Asif Khan Gillani]

I think this kind of solution will be easier to understand, document
and maintain. We can easily implement JPA without making entities for
everything. We just have to create entities to hold file content for
each kind of store.
Kindly guide me if there is any technical thing in adopting the
current file based mechanism for storage in DBs.

Post by John Blossom
It certainly would make sense. The LAMP stack will be there by default, and
if people want to use another SQL-serving mechanism they may. Hopefully this
would help to manage these state issues more flexibly, otherwise there's the
probability that the "simple" file solution becomes a bespoke database
solution that will be very troublesome to maintain.

[Yuri Z. (a.k.a Vega)]

I think it would be nice to make the Java (Derby) as default DB. That
would reduce minimal requirements to run Wiab.

Post by Asif Khan Gillani
I think this kind of solution will be easier to understand, document
and maintain. We can easily implement JPA without making entities for
everything. We just have to create entities to hold file content for
each kind of store.
Kindly guide me if there is any technical thing in adopting the
current file based mechanism for storage in DBs.

Post by John Blossom
It certainly would make sense. The LAMP stack will be there by default, and
if people want to use another SQL-serving mechanism they may. Hopefully this
would help to manage these state issues more flexibly, otherwise there's the
probability that the "simple" file solution becomes a bespoke database
solution that will be very troublesome to maintain.

[James Purser]

Personally if we're going to write a SQL driver, I think we're best of going
with the most commonly installed db - MySQL. Most developers would have it
installed, most hosting providers would have it and its a well understood
project.

James

Post by Yuri Z. (a.k.a Vega)
I think it would be nice to make the Java (Derby) as default DB. That
would reduce minimal requirements to run Wiab.

Post by Asif Khan Gillani
I think this kind of solution will be easier to understand, document
and maintain. We can easily implement JPA without making entities for
everything. We just have to create entities to hold file content for
each kind of store.
Kindly guide me if there is any technical thing in adopting the
current file based mechanism for storage in DBs.

Post by John Blossom
It certainly would make sense. The LAMP stack will be there by default,

and

Post by Asif Khan Gillani

Post by John Blossom
if people want to use another SQL-serving mechanism they may. Hopefully

this

Post by Asif Khan Gillani

Post by John Blossom
would help to manage these state issues more flexibly, otherwise

there's the

Post by Asif Khan Gillani

Post by John Blossom
probability that the "simple" file solution becomes a bespoke database
solution that will be very troublesome to maintain.

--
You received this message because you are subscribed to the Google Groups
"Wave Protocol" group.
To unsubscribe from this group, send email to
.
For more options, visit this group at
http://groups.google.com/group/wave-protocol?hl=en.

[Ya Knygar]

certainly, sql persistence would be a great leap forward,

but, since MySQL became Oracle's(with all respect, I think there MAY
be problems
with its free license or proper development)
and here were some interesting attempts with MongoDB(modern way)
i think its much more appropriate to choose by default PostgreSQL (so
many projects choose it now)
or , at least - MariaDB

I don't think there would be problems with finding PostgreSQL hosting
for people that want to implement WIAB , since - more or less
it still would be pretty heavy project, not situated for simple site
needs with
simple hosting plans but for VPS / VDS or so, were isn't
complications
to install any DB

so please, at least - make support for Postgres too, as ,
like you could see for now - it is, at least - very popular choice DB
for serious but OpSo
projects, and integrating with existing DB would be much easier for
many.

Post by John Blossom
It certainly would make sense. The LAMP stack will be there by default, and
if people want to use another SQL-serving mechanism they may. Hopefully this
would help to manage these state issues more flexibly, otherwise there's the
probability that the "simple" file solution becomes a bespoke database
solution that will be very troublesome to maintain.

[Yuri Z. (a.k.a Vega)]

I think that with proper data access abstraction - it shouldnt be a
problem to configure WIAB to work with any sql DB. But making the
default DB to be the JavaDB - would greatly reduce minimal
requirements - actually only Java runtime. Anyone would be able to run
Wiab on it's own local network.

Post by Ya Knygar
certainly, sql persistence would be a great leap forward,
but, since MySQL became Oracle's(with all respect, I think there MAY
be problems
with its free license or proper development)
and here were some interesting attempts with MongoDB(modern way)
i think its much more appropriate  to choose by default PostgreSQL (so
many projects choose it now)
or , at least - MariaDB
I don't think there would be problems with finding PostgreSQL hosting
for people that want to implement WIAB , since - more or less
it still would be pretty heavy project, not situated  for simple site
needs with
simple hosting plans but  for VPS / VDS or so, were  isn't
complications
to install any DB
so please, at least - make support for Postgres too, as ,
like you could see for now - it is, at least - very popular choice  DB
for serious but OpSo
projects,  and integrating with existing DB would be much easier for
many.

Post by John Blossom
It certainly would make sense. The LAMP stack will be there by default, and
if people want to use another SQL-serving mechanism they may. Hopefully this
would help to manage these state issues more flexibly, otherwise there's the
probability that the "simple" file solution becomes a bespoke database
solution that will be very troublesome to maintain.

[faisalbhagat]

can anyone guess til when we will be getting waveserver with delta
persistence in some db?

Post by Yuri Z. (a.k.a Vega)
I think that with proper data access abstraction - it shouldnt be a
problem to configure WIAB to work with any sql DB. But making the
default DB to be the JavaDB - would greatly reduce minimal
requirements - actually only Java runtime. Anyone would be able to run
Wiab on it's own local network.

Post by Ya Knygar
certainly, sql persistence would be a great leap forward,
but, since MySQL became Oracle's(with all respect, I think there MAY
be problems
with its free license or proper development)
and here were some interesting attempts with MongoDB(modern way)
i think its much more appropriate  to choose by default PostgreSQL (so
many projects choose it now)
or , at least - MariaDB
I don't think there would be problems with finding PostgreSQL hosting
for people that want to implement WIAB , since - more or less
it still would be pretty heavy project, not situated  for simple site
needs with
simple hosting plans but  for VPS / VDS or so, were  isn't
complications
to install any DB
so please, at least - make support for Postgres too, as ,
like you could see for now - it is, at least - very popular choice  DB
for serious but OpSo
projects,  and integrating with existing DB would be much easier for
many.

Post by John Blossom
It certainly would make sense. The LAMP stack will be there by default, and
if people want to use another SQL-serving mechanism they may. Hopefully this
would help to manage these state issues more flexibly, otherwise there's the
probability that the "simple" file solution becomes a bespoke database
solution that will be very troublesome to maintain.