Non-physical delete... again

[Andrey Razumovsky]

Hi,

We've discussed this time ago, but I'm back now with some ideas.

Current situation:
I have to use special boolean column named "deleted" in all DB tables. That
is, I never (or very rarely) do something like "DELETE FROM ..." or
"context.deleteObject()", instead I do "UPDATE... SET deleted=true" or
cdo.setDeleted(true). To select only undeleted records, I add qualifier like
"deleted=true" to every ObjEntity. Also I add pre-update listeners to all
entities which unregisters object and processes delete rules (also
respective to 'deleted' field) if 'deleted' flag is set.
It is obvious that Cayenne cannot completely handle this situation. One
example is CAY-1109, but it can be workarounded as described above. More
serious problems are:
- I cannot use properly flattened relationships, because 'deleted' qualifier
is not applied to DBEntities selection (e.g. in 'medium' table of
many-to-many relationship)
- flattened expressions, e.g. toArtist.name="Rublev" also miss that
qualifier
- there is no objects with DELETED state, only with MODIFIED
...and so on.

I faced this situation since the first days I used Cayenne. Now when I
thought about this a bit, I think I can handle it with two new features. I'm
not sure they completely match Cayenne paradigm, but...

1. DBEntity qualifier. This is same as ObjEntity qualifier, but applied
every time DBentity is being added to select sql. I will add DBEntity
qualifier 'deleted=false' and drop current ObjEntity's

2.  Deletion strategy for DataDomain. This means I do not always fire
DeleteAction, but maybe something else (update action in my case).

Any comments much appreciated.

Thanks,
Andrey

[Mike Kienenberger]

I have a legacy Cayenne 1.1.4 app that does the same thing.
I handled it back then by doing something similar.   Back then, we
could set up a delegate to intercept all queries, and I automatically
added the restricting qualifier to each select query.

I never deleted from the application level code .  Instead, I had a
special invalidation manager class that I called instead which would
update the boolean field (and a timestamp) and invalidate all objects
referencing that object.

It worked out well enough, although there were issues with the
invalidation propagating to all sessions exactly when I wanted it to
happen.

However, I didn't have any flattened relationships back then -- too
ignorant to know any better.


On Mon, Jun 1, 2009 at 11:09 AM, Andrey Razumovsky
<[hidden email]> wrote:

> Hi,
>
> We've discussed this time ago, but I'm back now with some ideas.
>
> Current situation:
> I have to use special boolean column named "deleted" in all DB tables. That
> is, I never (or very rarely) do something like "DELETE FROM ..." or
> "context.deleteObject()", instead I do "UPDATE... SET deleted=true" or
> cdo.setDeleted(true). To select only undeleted records, I add qualifier like
> "deleted=true" to every ObjEntity. Also I add pre-update listeners to all
> entities which unregisters object and processes delete rules (also
> respective to 'deleted' field) if 'deleted' flag is set.
> It is obvious that Cayenne cannot completely handle this situation. One
> example is CAY-1109, but it can be workarounded as described above. More
> serious problems are:
> - I cannot use properly flattened relationships, because 'deleted' qualifier
> is not applied to DBEntities selection (e.g. in 'medium' table of
> many-to-many relationship)
> - flattened expressions, e.g. toArtist.name="Rublev" also miss that
> qualifier
> - there is no objects with DELETED state, only with MODIFIED
> ...and so on.
>
> I faced this situation since the first days I used Cayenne. Now when I
> thought about this a bit, I think I can handle it with two new features. I'm
> not sure they completely match Cayenne paradigm, but...
>
> 1. DBEntity qualifier. This is same as ObjEntity qualifier, but applied
> every time DBentity is being added to select sql. I will add DBEntity
> qualifier 'deleted=false' and drop current ObjEntity's
>
> 2.  Deletion strategy for DataDomain. This means I do not always fire
> DeleteAction, but maybe something else (update action in my case).
>
> Any comments much appreciated.
>
> Thanks,
> Andrey
>

[Andrus Adamchik]

On Jun 1, 2009, at 6:09 PM, Andrey Razumovsky wrote:
>
>
> 2.  Deletion strategy for DataDomain. This means I do not always fire
> DeleteAction, but maybe something else (update action in my case).

Yeah, good idea to move this down the stack. I would love to have the  
ObjectContext level code to stay unchanged, but then generate UPDATE  
instead of DELETE at the lower levels if entity is tagged as "soft  
delete".

> 1. DBEntity qualifier. This is same as ObjEntity qualifier, but  
> applied
> every time DBentity is being added to select sql. I will add DBEntity
> qualifier 'deleted=false' and drop current ObjEntity's

So does that mean dropping ObjEntity qualifiers completely? (This may  
or may not be a good idea, I am not yet sure myself).

Also I think there's a possibility here to address another annoyance  
in one shot - the need to manually set the inheritance discriminator  
column for new objects when inheritance is used. If we can add  
functionality to expressions to not only "get" the value from an  
object, but also "set" the value, we can do both soft delete and auto  
inheritance update based on an (almost) arbitrary exception. Kind of  
like OGNL framework does...

Andrus

[Andrey Razumovsky]

Hi,

> Back then, we
> could set up a delegate to intercept all queries, and I automatically
> added the restricting qualifier to each select query.

I'm afraid, simple qualifier would not be enough, I need to set them for all
joined tables in query (I have simple qualifirer already, by using ObjEntity
qualifier).

2009/6/1 Andrus Adamchik <[hidden email]>

>
>
> On Jun 1, 2009, at 6:09 PM, Andrey Razumovsky wrote:
>
>>
>>
>> 2.  Deletion strategy for DataDomain. This means I do not always fire
>> DeleteAction, but maybe something else (update action in my case).
>>
>
> Yeah, good idea to move this down the stack. I would love to have the
> ObjectContext level code to stay unchanged, but then generate UPDATE instead
> of DELETE at the lower levels if entity is tagged as "soft delete".

I actually want to see this more generic, i.e. DeletionStrategy interface
with one default implementation (which fires DELETE) and ability to set my
own.


>
>
>  1. DBEntity qualifier. This is same as ObjEntity qualifier, but applied
>> every time DBentity is being added to select sql. I will add DBEntity
>> qualifier 'deleted=false' and drop current ObjEntity's
>>
>
> So does that mean dropping ObjEntity qualifiers completely? (This may or
> may not be a good idea, I am not yet sure myself).


I mean removing them in my Cayenne project of course, not in API.


> Also I think there's a possibility here to address another annoyance in one
> shot - the need to manually set the inheritance discriminator column for new
> objects when inheritance is used. If we can add functionality to expressions
> to not only "get" the value from an object, but also "set" the value, we can
> do both soft delete and auto inheritance update based on an (almost)
> arbitrary exception. Kind of like OGNL framework does...
>
>
Well, I agree this is must-have in Cayenne. But my proposed solution is a
separate feature, because I need intercepting at DbEntity level.

If noone minds against these features, I think I'll look at implementing
'DeleteStrategy' and 'DbEntity qualifiers'

Andrey

[Aristedes Maniatis]

On 02/06/2009, at 7:08 PM, Andrey Razumovsky wrote:

>> Yeah, good idea to move this down the stack. I would love to have the
>> ObjectContext level code to stay unchanged, but then generate  
>> UPDATE instead
>> of DELETE at the lower levels if entity is tagged as "soft delete".
>
>
> I actually want to see this more generic, i.e. DeletionStrategy  
> interface
> with one default implementation (which fires DELETE) and ability to  
> set my
> own.

Two thoughts:

1. Because the second most commonly used implementation will be:

* boolean NOT NULL field called 'isDeleted'

Could this alternative implementation be one of the 'out of the box'  
choices? For us, we'd want that choice per table since some tables we  
actually do delete (since we don't care about keeping historical  
data). For Cayenne, the ability to do this without too much code would  
be very useful and a strong 'selling point' for the framework.



2. This is only a step from having an update strategy as well. For  
instance, a customer who wanted to keep a complete audit trail of all  
changes to Contact records might want every UPDATE to actually be an  
INSERT with the old record marked as deleted and all relations  
appropriately adjusted. I'm not convinced this is the best design  
pattern (as opposed to storing the diffs of the changes in some log  
file), but I've been thinking a little about this approach.




Ari


-------------------------->
ish
http://www.ish.com.au
Level 1, 30 Wilson Street Newtown 2042 Australia
phone [phone]   fax [phone]
GPG fingerprint CBFB 84B4 738D 4E87 5E5C  5EFA EF6A 7D2E 3E49 102A

[Andrus Adamchik]

On Jun 2, 2009, at 12:08 PM, Andrey Razumovsky wrote:

>>>
>>> 2.  Deletion strategy for DataDomain. This means I do not always  
>>> fire
>>> DeleteAction, but maybe something else (update action in my case).
>>>
>>
>> Yeah, good idea to move this down the stack. I would love to have the
>> ObjectContext level code to stay unchanged, but then generate  
>> UPDATE instead
>> of DELETE at the lower levels if entity is tagged as "soft delete".
>
>
> I actually want to see this more generic, i.e. DeletionStrategy  
> interface
> with one default implementation (which fires DELETE) and ability to  
> set my
> own.

Not sure I understand, but I guess we'll see how it looks when you  
commit the code.

Just one question - are you planning to implement a "soft delete"  
feature in Cayenne (with appropriate flags per entity settable in the  
Modeler), or just an extension point to hook up your own strategy (or  
both)?

Andrus

[Andrus Adamchik]

On Jun 2, 2009, at 12:29 PM, Aristedes Maniatis wrote:

>
> 1. Because the second most commonly used implementation will be:
>
> * boolean NOT NULL field called 'isDeleted'
>
> Could this alternative implementation be one of the 'out of the box'  
> choices? For us, we'd want that choice per table since some tables  
> we actually do delete (since we don't care about keeping historical  
> data). For Cayenne, the ability to do this without too much code  
> would be very useful and a strong 'selling point' for the framework.

(BTW we have another area where we could provide a sensible default -  
the auto increment version column.)

What I would like to avoid is implicit invisible mappings forcing user  
into Cayenne table/column naming conventions. The ability to work on  
top of an (almost) arbitrary schema is a big deal in the database  
world. Cayenne never had any such conventions (ok, except for  
AUTO_PK_TABLE), and this was also a strong usability point.

Fortunately we have the Modeler, so we can achieve the best of both  
worlds. The modeler can create a needed column and a qualifier if an  
entity is marked as "soft delete". The user can stick with those, or  
go and edit the names.

Andrus

[Aristedes Maniatis]

On 02/06/2009, at 8:08 PM, Andrus Adamchik wrote:

>
> On Jun 2, 2009, at 12:29 PM, Aristedes Maniatis wrote:
>
>>
>> 1. Because the second most commonly used implementation will be:
>>
>> * boolean NOT NULL field called 'isDeleted'
>>
>> Could this alternative implementation be one of the 'out of the  
>> box' choices? For us, we'd want that choice per table since some  
>> tables we actually do delete (since we don't care about keeping  
>> historical data). For Cayenne, the ability to do this without too  
>> much code would be very useful and a strong 'selling point' for the  
>> framework.
>
> (BTW we have another area where we could provide a sensible default  
> - the auto increment version column.)
>
> What I would like to avoid is implicit invisible mappings forcing  
> user into Cayenne table/column naming conventions. The ability to  
> work on top of an (almost) arbitrary schema is a big deal in the  
> database world. Cayenne never had any such conventions (ok, except  
> for AUTO_PK_TABLE), and this was also a strong usability point.
>
> Fortunately we have the Modeler, so we can achieve the best of both  
> worlds. The modeler can create a needed column and a qualifier if an  
> entity is marked as "soft delete". The user can stick with those, or  
> go and edit the names.

For sure. I think Andrey wants to implement a generic hook so that the  
entire implementation of what 'delete' means can be replaced by an  
alternate strategy almost like specifying a lifecycle listener. That  
sounds really complex for a user to implement, but some baked-in  
strategies would be useful. You would still get to choose which column  
represents the concept of 'deleted', just as you currently choose  
which column is the inheritance discriminator in the superclass  
dbEntity.

Ari



-------------------------->
ish
http://www.ish.com.au
Level 1, 30 Wilson Street Newtown 2042 Australia
phone [phone]   fax [phone]
GPG fingerprint CBFB 84B4 738D 4E87 5E5C  5EFA EF6A 7D2E 3E49 102A

[Andrey Razumovsky]

I plan to add some default 'soft' strategy for everyone to use, and it will
likely have name of 'deleted' field as constructor parameter. But I'm no fan
of adding some sort of 'soft' checkbox for dbattributes, because it will
make new problems, such as handling several soft attributes and so on. This
is quite a rare case, after all. Modeler support will be covered by setting
class name of strategy

2009/6/2 Aristedes Maniatis <[hidden email]>

>
> On 02/06/2009, at 8:08 PM, Andrus Adamchik wrote:
>
>
>> On Jun 2, 2009, at 12:29 PM, Aristedes Maniatis wrote:
>>
>>
>>> 1. Because the second most commonly used implementation will be:
>>>
>>> * boolean NOT NULL field called 'isDeleted'
>>>
>>> Could this alternative implementation be one of the 'out of the box'
>>> choices? For us, we'd want that choice per table since some tables we
>>> actually do delete (since we don't care about keeping historical data). For
>>> Cayenne, the ability to do this without too much code would be very useful
>>> and a strong 'selling point' for the framework.
>>>
>>
>> (BTW we have another area where we could provide a sensible default - the
>> auto increment version column.)
>>
>> What I would like to avoid is implicit invisible mappings forcing user
>> into Cayenne table/column naming conventions. The ability to work on top of
>> an (almost) arbitrary schema is a big deal in the database world. Cayenne
>> never had any such conventions (ok, except for AUTO_PK_TABLE), and this was
>> also a strong usability point.
>>
>> Fortunately we have the Modeler, so we can achieve the best of both
>> worlds. The modeler can create a needed column and a qualifier if an entity
>> is marked as "soft delete". The user can stick with those, or go and edit
>> the names.
>>
>
>
> For sure. I think Andrey wants to implement a generic hook so that the
> entire implementation of what 'delete' means can be replaced by an alternate
> strategy almost like specifying a lifecycle listener. That sounds really
> complex for a user to implement, but some baked-in strategies would be
> useful. You would still get to choose which column represents the concept of
> 'deleted', just as you currently choose which column is the inheritance
> discriminator in the superclass dbEntity.
>
>
> Ari
>
>
>
> -------------------------->
> ish
> http://www.ish.com.au
> Level 1, 30 Wilson Street Newtown 2042 Australia
> phone [phone]   fax [phone]
> GPG fingerprint CBFB 84B4 738D 4E87 5E5C  5EFA EF6A 7D2E 3E49 102A
>
>
>

[Andrus Adamchik]

On Jun 2, 2009, at 2:54 PM, Andrey Razumovsky wrote:

> But I'm no fan of adding some sort of 'soft' checkbox for dbattributes

I was suggesting marking entity with a "soft delete" checkbox (not  
individual attribute - this would make no sense), and creating a  
criteria based on qualifier that references an attribute.

> Modeler support will be covered by setting class name of strategy

I am afraid this approach will be rather arbitrary to the end user, so  
I suggest we discuss it some more before putting it in Cayenne.  
Marking an entity to use "soft delete" based on some criteria is a  
clear and understandable feature. Setting a "delete strategy" is not,  
and will contribute to confusion. This is totally be ok as a backend  
extension point, but I will hate to see that as a general use feature.

Andrus

[Andrus Adamchik]

On Jun 2, 2009, at 4:38 PM, Andrus Adamchik wrote:

>>
>> Modeler support will be covered by setting class name of strategy
>
> I am afraid this approach will be rather arbitrary to the end user,  
> so I suggest we discuss it some more before putting it in Cayenne.  
> Marking an entity to use "soft delete" based on some criteria is a  
> clear and understandable feature. Setting a "delete strategy" is  
> not, and will contribute to confusion. This is totally be ok as a  
> backend extension point, but I will hate to see that as a general  
> use feature.

In this context let me mention one idea for Cayenne 3.0 + N, that I've  
been thinking about for some time. I am taking this to a separate  
thread to avoid distraction from the soft delete discussion, which has  
only tangential relevance.

Since we already have a bunch of extension points throughout the  
stack, some exposed via the Modeler (misplaced like cache JGroups  
config, or justified like Adapter config), and some are available only  
via the code, we need a way to reign them in. The standard way of  
doing that is via an IoC container.

No, I don't want to bundle Spring with Cayenne, besides it has to  
integrate with the larger app ecosystem, so we still need to figure  
the technical details. But the point is that we will be able to  
provide a single place to configure all extension points, separate  
from the mapping. As unlike the mapping those parameters are often  
different for the same project, depending on the environment where it  
is deployed.

Right now this place is cayenne.xml (and it might as well stay this  
way in the future), just that unlike say Spring config files, it has a  
rigid structure and is not generic enough to handle arbitrary  
extensions and dependencies. It was ok for the early versions of  
Cayenne, since there was only a few things you could change (data  
source factory and adapter I believe). But now something more powerful  
and clean is desirable.

Just some raw thoughts.

Andrus

[Andrey Razumovsky]

For not to be unsubstantiated, I uploaded my vision of the feature. Note
that there is no modeler support. It allows to provide custom factory that
creates builders of INSERT, UPDATE, DELETE queries. 'Soft-delete' factory is
included.
I do not say that 'soft delete' checkbox is not needed, but uploaded code is
much more generic and allows to plug any behavior (maybe even Ari's proposed
'versions'). 'Soft' strategy can be configured with 'deleted' field name and
will not fire UPDATE if such field does not exist in DB table.

Please add your comments!

Thanks,
Andrey

2009/6/2 Andrus Adamchik <[hidden email]>

>
> On Jun 2, 2009, at 2:54 PM, Andrey Razumovsky wrote:
>
>  But I'm no fan of adding some sort of 'soft' checkbox for dbattributes
>>
>
> I was suggesting marking entity with a "soft delete" checkbox (not
> individual attribute - this would make no sense), and creating a criteria
> based on qualifier that references an attribute.
>
>  Modeler support will be covered by setting class name of strategy
>>
>
> I am afraid this approach will be rather arbitrary to the end user, so I
> suggest we discuss it some more before putting it in Cayenne. Marking an
> entity to use "soft delete" based on some criteria is a clear and
> understandable feature. Setting a "delete strategy" is not, and will
> contribute to confusion. This is totally be ok as a backend extension point,
> but I will hate to see that as a general use feature.
>
> Andrus
>
>
>

[Andrus Adamchik]

Excellent. I have one comment (consistent with my position in the  
previous message). Configuration of the factory below should be  
attached to the stack objects (e.g. DataDomain), not the mapping  
objects (DataMap):

+
+    /**
+     * Sets factory for creating QueryBuilders
+     */
+    public void setQueryBuilderFactory(BatchQueryBuilderFactory  
queryBuilderFactory) {
+        this.queryBuilderFactory = queryBuilderFactory;
+    }
+
+    /**
+     * @return factory for creating QueryBuilders. Might be null
+     */
+    public BatchQueryBuilderFactory getQueryBuilderFactory() {
+        return queryBuilderFactory;
+    }

Cheers,
Andrus

P.S. Maybe you can separately commit the @Deprecated part of the  
patch ;-)


On Jun 2, 2009, at 5:07 PM, Andrey Razumovsky wrote:

> For not to be unsubstantiated, I uploaded my vision of the feature.  
> Note
> that there is no modeler support. It allows to provide custom  
> factory that
> creates builders of INSERT, UPDATE, DELETE queries. 'Soft-delete'  
> factory is
> included.
> I do not say that 'soft delete' checkbox is not needed, but uploaded  
> code is
> much more generic and allows to plug any behavior (maybe even Ari's  
> proposed
> 'versions'). 'Soft' strategy can be configured with 'deleted' field  
> name and
> will not fire UPDATE if such field does not exist in DB table.
>
> Please add your comments!
>
> Thanks,
> Andrey
>
> 2009/6/2 Andrus Adamchik <[hidden email]>
>
>>
>> On Jun 2, 2009, at 2:54 PM, Andrey Razumovsky wrote:
>>
>> But I'm no fan of adding some sort of 'soft' checkbox for  
>> dbattributes
>>>
>>
>> I was suggesting marking entity with a "soft delete" checkbox (not
>> individual attribute - this would make no sense), and creating a  
>> criteria
>> based on qualifier that references an attribute.
>>
>> Modeler support will be covered by setting class name of strategy
>>>
>>
>> I am afraid this approach will be rather arbitrary to the end user,  
>> so I
>> suggest we discuss it some more before putting it in Cayenne.  
>> Marking an
>> entity to use "soft delete" based on some criteria is a clear and
>> understandable feature. Setting a "delete strategy" is not, and will
>> contribute to confusion. This is totally be ok as a backend  
>> extension point,
>> but I will hate to see that as a general use feature.
>>
>> Andrus
>>
>>
>>

[Andrey Razumovsky]

I swiched to DataMap after I saw no quick access to DataDomain in
BatchAction class. This needs futher investigation, we need to pass domain
there somehow. I assume API changes in public classes of access.jdbc package
are allowed?

2009/6/2 Andrus Adamchik <[hidden email]>

> Excellent. I have one comment (consistent with my position in the previous
> message). Configuration of the factory below should be attached to the stack
> objects (e.g. DataDomain), not the mapping objects (DataMap):
>
> +
> +    /**
> +     * Sets factory for creating QueryBuilders
> +     */
> +    public void setQueryBuilderFactory(BatchQueryBuilderFactory
> queryBuilderFactory) {
> +        this.queryBuilderFactory = queryBuilderFactory;
> +    }
> +
> +    /**
> +     * @return factory for creating QueryBuilders. Might be null
> +     */
> +    public BatchQueryBuilderFactory getQueryBuilderFactory() {
> +        return queryBuilderFactory;
> +    }
>
> Cheers,
> Andrus
>
> P.S. Maybe you can separately commit the @Deprecated part of the patch ;-)
>
>
>
> On Jun 2, 2009, at 5:07 PM, Andrey Razumovsky wrote:
>
>  For not to be unsubstantiated, I uploaded my vision of the feature. Note
>> that there is no modeler support. It allows to provide custom factory that
>> creates builders of INSERT, UPDATE, DELETE queries. 'Soft-delete' factory
>> is
>> included.
>> I do not say that 'soft delete' checkbox is not needed, but uploaded code
>> is
>> much more generic and allows to plug any behavior (maybe even Ari's
>> proposed
>> 'versions'). 'Soft' strategy can be configured with 'deleted' field name
>> and
>> will not fire UPDATE if such field does not exist in DB table.
>>
>> Please add your comments!
>>
>> Thanks,
>> Andrey
>>
>> 2009/6/2 Andrus Adamchik <[hidden email]>
>>
>>
>>> On Jun 2, 2009, at 2:54 PM, Andrey Razumovsky wrote:
>>>
>>> But I'm no fan of adding some sort of 'soft' checkbox for dbattributes
>>>
>>>>
>>>>
>>> I was suggesting marking entity with a "soft delete" checkbox (not
>>> individual attribute - this would make no sense), and creating a criteria
>>> based on qualifier that references an attribute.
>>>
>>> Modeler support will be covered by setting class name of strategy
>>>
>>>>
>>>>
>>> I am afraid this approach will be rather arbitrary to the end user, so I
>>> suggest we discuss it some more before putting it in Cayenne. Marking an
>>> entity to use "soft delete" based on some criteria is a clear and
>>> understandable feature. Setting a "delete strategy" is not, and will
>>> contribute to confusion. This is totally be ok as a backend extension
>>> point,
>>> but I will hate to see that as a general use feature.
>>>
>>> Andrus
>>>
>>>
>>>
>>>
>

[Andrus Adamchik]

> I swiched to DataMap after I saw no quick access to DataDomain in
> BatchAction class.

It all starts with DataDomainFlushAction, which has a reference to DD,  
so there should be a way to inject factory down the tree.

> I assume API changes in public classes of access.jdbc package are  
> allowed?

Until we are in Beta, yes.

Andrus


On Jun 2, 2009, at 5:25 PM, Andrey Razumovsky wrote:

> I swiched to DataMap after I saw no quick access to DataDomain in
> BatchAction class. This needs futher investigation, we need to pass  
> domain
> there somehow. I assume API changes in public classes of access.jdbc  
> package
> are allowed?
>
> 2009/6/2 Andrus Adamchik <[hidden email]>
>
>> Excellent. I have one comment (consistent with my position in the  
>> previous
>> message). Configuration of the factory below should be attached to  
>> the stack
>> objects (e.g. DataDomain), not the mapping objects (DataMap):
>>
>> +
>> +    /**
>> +     * Sets factory for creating QueryBuilders
>> +     */
>> +    public void setQueryBuilderFactory(BatchQueryBuilderFactory
>> queryBuilderFactory) {
>> +        this.queryBuilderFactory = queryBuilderFactory;
>> +    }
>> +
>> +    /**
>> +     * @return factory for creating QueryBuilders. Might be null
>> +     */
>> +    public BatchQueryBuilderFactory getQueryBuilderFactory() {
>> +        return queryBuilderFactory;
>> +    }
>>
>> Cheers,
>> Andrus
>>
>> P.S. Maybe you can separately commit the @Deprecated part of the  
>> patch ;-)
>>
>>
>>
>> On Jun 2, 2009, at 5:07 PM, Andrey Razumovsky wrote:
>>
>> For not to be unsubstantiated, I uploaded my vision of the feature.  
>> Note
>>> that there is no modeler support. It allows to provide custom  
>>> factory that
>>> creates builders of INSERT, UPDATE, DELETE queries. 'Soft-delete'  
>>> factory
>>> is
>>> included.
>>> I do not say that 'soft delete' checkbox is not needed, but  
>>> uploaded code
>>> is
>>> much more generic and allows to plug any behavior (maybe even Ari's
>>> proposed
>>> 'versions'). 'Soft' strategy can be configured with 'deleted'  
>>> field name
>>> and
>>> will not fire UPDATE if such field does not exist in DB table.
>>>
>>> Please add your comments!
>>>
>>> Thanks,
>>> Andrey
>>>
>>> 2009/6/2 Andrus Adamchik <[hidden email]>
>>>
>>>
>>>> On Jun 2, 2009, at 2:54 PM, Andrey Razumovsky wrote:
>>>>
>>>> But I'm no fan of adding some sort of 'soft' checkbox for  
>>>> dbattributes
>>>>
>>>>>
>>>>>
>>>> I was suggesting marking entity with a "soft delete" checkbox (not
>>>> individual attribute - this would make no sense), and creating a  
>>>> criteria
>>>> based on qualifier that references an attribute.
>>>>
>>>> Modeler support will be covered by setting class name of strategy
>>>>
>>>>>
>>>>>
>>>> I am afraid this approach will be rather arbitrary to the end  
>>>> user, so I
>>>> suggest we discuss it some more before putting it in Cayenne.  
>>>> Marking an
>>>> entity to use "soft delete" based on some criteria is a clear and
>>>> understandable feature. Setting a "delete strategy" is not, and  
>>>> will
>>>> contribute to confusion. This is totally be ok as a backend  
>>>> extension
>>>> point,
>>>> but I will hate to see that as a general use feature.
>>>>
>>>> Andrus
>>>>
>>>>
>>>>
>>>>
>>

[Robert Zeigler-6]

If you're really considering going the 3rd party ioc route, I highly  
recommend T5IOC.
Note that configuration is (typically) done via code in T5IOC, but I  
find it extremely flexible & powerful, while still being simple to use  
(and small! :).
If not that, then guice. I'd even go for pico (though preferably  
not).  Anything but the monster that spring has become. ;)

Robert

On Jun 2, 2009, at 6/29:02 AM , Andrus Adamchik wrote:

>
> On Jun 2, 2009, at 4:38 PM, Andrus Adamchik wrote:
>
>>>
>>> Modeler support will be covered by setting class name of strategy
>>
>> I am afraid this approach will be rather arbitrary to the end user,  
>> so I suggest we discuss it some more before putting it in Cayenne.  
>> Marking an entity to use "soft delete" based on some criteria is a  
>> clear and understandable feature. Setting a "delete strategy" is  
>> not, and will contribute to confusion. This is totally be ok as a  
>> backend extension point, but I will hate to see that as a general  
>> use feature.
>
> In this context let me mention one idea for Cayenne 3.0 + N, that  
> I've been thinking about for some time. I am taking this to a  
> separate thread to avoid distraction from the soft delete  
> discussion, which has only tangential relevance.
>
> Since we already have a bunch of extension points throughout the  
> stack, some exposed via the Modeler (misplaced like cache JGroups  
> config, or justified like Adapter config), and some are available  
> only via the code, we need a way to reign them in. The standard way  
> of doing that is via an IoC container.
>
> No, I don't want to bundle Spring with Cayenne, besides it has to  
> integrate with the larger app ecosystem, so we still need to figure  
> the technical details. But the point is that we will be able to  
> provide a single place to configure all extension points, separate  
> from the mapping. As unlike the mapping those parameters are often  
> different for the same project, depending on the environment where  
> it is deployed.
>
> Right now this place is cayenne.xml (and it might as well stay this  
> way in the future), just that unlike say Spring config files, it has  
> a rigid structure and is not generic enough to handle arbitrary  
> extensions and dependencies. It was ok for the early versions of  
> Cayenne, since there was only a few things you could change (data  
> source factory and adapter I believe). But now something more  
> powerful and clean is desirable.
>
> Just some raw thoughts.
>
> Andrus

[Andrus Adamchik]

I have a good opinion about Tapestry IoC approach in general  
(including the now defunct Hivemind), and I wanted to investigate Guice.

There's some conflicting requirements to address here - we don't want  
to write/maintain our own IoC container, yet, we don't want to embed a  
huge third-party engine, of which we'll use only a subset of features.  
I'd like it to work standalone, as well as be able to integrate (or at  
least play well) with popular IoC containers (how many containers in  
one app is too many?). Then there's a matter of modeler support, which  
is adverse to annotations, and favors XML or other config files...

All in all, I think assembling a core of Cayenne stack via such a  
container should open some interesting possibilities, beyond  
organizing current configuration.

Andrus



On Jun 2, 2009, at 6:53 PM, Robert Zeigler wrote:

> If you're really considering going the 3rd party ioc route, I highly  
> recommend T5IOC.
> Note that configuration is (typically) done via code in T5IOC, but I  
> find it extremely flexible & powerful, while still being simple to  
> use (and small! :).
> If not that, then guice. I'd even go for pico (though preferably  
> not).  Anything but the monster that spring has become. ;)
>
> Robert
>
> On Jun 2, 2009, at 6/29:02 AM , Andrus Adamchik wrote:
>
>>
>> On Jun 2, 2009, at 4:38 PM, Andrus Adamchik wrote:
>>
>>>>
>>>> Modeler support will be covered by setting class name of strategy
>>>
>>> I am afraid this approach will be rather arbitrary to the end  
>>> user, so I suggest we discuss it some more before putting it in  
>>> Cayenne. Marking an entity to use "soft delete" based on some  
>>> criteria is a clear and understandable feature. Setting a "delete  
>>> strategy" is not, and will contribute to confusion. This is  
>>> totally be ok as a backend extension point, but I will hate to see  
>>> that as a general use feature.
>>
>> In this context let me mention one idea for Cayenne 3.0 + N, that  
>> I've been thinking about for some time. I am taking this to a  
>> separate thread to avoid distraction from the soft delete  
>> discussion, which has only tangential relevance.
>>
>> Since we already have a bunch of extension points throughout the  
>> stack, some exposed via the Modeler (misplaced like cache JGroups  
>> config, or justified like Adapter config), and some are available  
>> only via the code, we need a way to reign them in. The standard way  
>> of doing that is via an IoC container.
>>
>> No, I don't want to bundle Spring with Cayenne, besides it has to  
>> integrate with the larger app ecosystem, so we still need to figure  
>> the technical details. But the point is that we will be able to  
>> provide a single place to configure all extension points, separate  
>> from the mapping. As unlike the mapping those parameters are often  
>> different for the same project, depending on the environment where  
>> it is deployed.
>>
>> Right now this place is cayenne.xml (and it might as well stay this  
>> way in the future), just that unlike say Spring config files, it  
>> has a rigid structure and is not generic enough to handle arbitrary  
>> extensions and dependencies. It was ok for the early versions of  
>> Cayenne, since there was only a few things you could change (data  
>> source factory and adapter I believe). But now something more  
>> powerful and clean is desirable.
>>
>> Just some raw thoughts.
>>
>> Andrus
>
>

[Andrey Razumovsky]

It's not that easy as it sounds. BatchAction is born from BatchQuery (which
is in .query package and does not have access to DataDomain/Node) and
JDBCActionBuilder (which is logical part of JdbcAdapter, and also does not
have access to DataDomain/Node). After all, batch queries are public and can
be fired from anywhere, including context.performQuery(..). So it starts not
with DataDomainFlushAction, but with DataNodeQueryAction. The best I could
do was adding factory property to DataNode and passing it to action in query
observer.

2009/6/2 Andrus Adamchik <[hidden email]>

> I swiched to DataMap after I saw no quick access to DataDomain in
>> BatchAction class.
>>
>
> It all starts with DataDomainFlushAction, which has a reference to DD, so
> there should be a way to inject factory down the tree.
>
>  I assume API changes in public classes of access.jdbc package are allowed?
>>
>
> Until we are in Beta, yes.
>
> Andrus
>
>
>
> On Jun 2, 2009, at 5:25 PM, Andrey Razumovsky wrote:
>
>  I swiched to DataMap after I saw no quick access to DataDomain in
>> BatchAction class. This needs futher investigation, we need to pass domain
>> there somehow. I assume API changes in public classes of access.jdbc
>> package
>> are allowed?
>>
>> 2009/6/2 Andrus Adamchik <[hidden email]>
>>
>>  Excellent. I have one comment (consistent with my position in the
>>> previous
>>> message). Configuration of the factory below should be attached to the
>>> stack
>>> objects (e.g. DataDomain), not the mapping objects (DataMap):
>>>
>>> +
>>> +    /**
>>> +     * Sets factory for creating QueryBuilders
>>> +     */
>>> +    public void setQueryBuilderFactory(BatchQueryBuilderFactory
>>> queryBuilderFactory) {
>>> +        this.queryBuilderFactory = queryBuilderFactory;
>>> +    }
>>> +
>>> +    /**
>>> +     * @return factory for creating QueryBuilders. Might be null
>>> +     */
>>> +    public BatchQueryBuilderFactory getQueryBuilderFactory() {
>>> +        return queryBuilderFactory;
>>> +    }
>>>
>>> Cheers,
>>> Andrus
>>>
>>> P.S. Maybe you can separately commit the @Deprecated part of the patch
>>> ;-)
>>>
>>>
>>>
>>> On Jun 2, 2009, at 5:07 PM, Andrey Razumovsky wrote:
>>>
>>> For not to be unsubstantiated, I uploaded my vision of the feature. Note
>>>
>>>> that there is no modeler support. It allows to provide custom factory
>>>> that
>>>> creates builders of INSERT, UPDATE, DELETE queries. 'Soft-delete'
>>>> factory
>>>> is
>>>> included.
>>>> I do not say that 'soft delete' checkbox is not needed, but uploaded
>>>> code
>>>> is
>>>> much more generic and allows to plug any behavior (maybe even Ari's
>>>> proposed
>>>> 'versions'). 'Soft' strategy can be configured with 'deleted' field name
>>>> and
>>>> will not fire UPDATE if such field does not exist in DB table.
>>>>
>>>> Please add your comments!
>>>>
>>>> Thanks,
>>>> Andrey
>>>>
>>>> 2009/6/2 Andrus Adamchik <[hidden email]>
>>>>
>>>>
>>>>  On Jun 2, 2009, at 2:54 PM, Andrey Razumovsky wrote:
>>>>>
>>>>> But I'm no fan of adding some sort of 'soft' checkbox for dbattributes
>>>>>
>>>>>
>>>>>>
>>>>>>  I was suggesting marking entity with a "soft delete" checkbox (not
>>>>> individual attribute - this would make no sense), and creating a
>>>>> criteria
>>>>> based on qualifier that references an attribute.
>>>>>
>>>>> Modeler support will be covered by setting class name of strategy
>>>>>
>>>>>
>>>>>>
>>>>>>  I am afraid this approach will be rather arbitrary to the end user,
>>>>> so I
>>>>> suggest we discuss it some more before putting it in Cayenne. Marking
>>>>> an
>>>>> entity to use "soft delete" based on some criteria is a clear and
>>>>> understandable feature. Setting a "delete strategy" is not, and will
>>>>> contribute to confusion. This is totally be ok as a backend extension
>>>>> point,
>>>>> but I will hate to see that as a general use feature.
>>>>>
>>>>> Andrus
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>
>

[Andrus Adamchik]

>  After all, batch queries are public and can
> be fired from anywhere, including context.performQuery(..).

Exactly, BatchQuery is just that - a query. And we probably do not  
want to redefine how it is processed. So this leads me to think that  
the logic fork should still happen somewhere inside  
DataDomainFlushAction. You'd generate a BatchUpdateAction instead of  
delete or something like that.

Andrus


On Jun 3, 2009, at 11:16 AM, Andrey Razumovsky wrote:

> It's not that easy as it sounds. BatchAction is born from BatchQuery  
> (which
> is in .query package and does not have access to DataDomain/Node) and
> JDBCActionBuilder (which is logical part of JdbcAdapter, and also  
> does not
> have access to DataDomain/Node). After all, batch queries are public  
> and can
> be fired from anywhere, including context.performQuery(..). So it  
> starts not
> with DataDomainFlushAction, but with DataNodeQueryAction. The best I  
> could
> do was adding factory property to DataNode and passing it to action  
> in query
> observer.
>
> 2009/6/2 Andrus Adamchik <[hidden email]>
>
>> I swiched to DataMap after I saw no quick access to DataDomain in
>>> BatchAction class.
>>>
>>
>> It all starts with DataDomainFlushAction, which has a reference to  
>> DD, so
>> there should be a way to inject factory down the tree.
>>
>> I assume API changes in public classes of access.jdbc package are  
>> allowed?
>>>
>>
>> Until we are in Beta, yes.
>>
>> Andrus
>>
>>
>>
>> On Jun 2, 2009, at 5:25 PM, Andrey Razumovsky wrote:
>>
>> I swiched to DataMap after I saw no quick access to DataDomain in
>>> BatchAction class. This needs futher investigation, we need to  
>>> pass domain
>>> there somehow. I assume API changes in public classes of access.jdbc
>>> package
>>> are allowed?
>>>
>>> 2009/6/2 Andrus Adamchik <[hidden email]>
>>>
>>> Excellent. I have one comment (consistent with my position in the
>>>> previous
>>>> message). Configuration of the factory below should be attached  
>>>> to the
>>>> stack
>>>> objects (e.g. DataDomain), not the mapping objects (DataMap):
>>>>
>>>> +
>>>> +    /**
>>>> +     * Sets factory for creating QueryBuilders
>>>> +     */
>>>> +    public void setQueryBuilderFactory(BatchQueryBuilderFactory
>>>> queryBuilderFactory) {
>>>> +        this.queryBuilderFactory = queryBuilderFactory;
>>>> +    }
>>>> +
>>>> +    /**
>>>> +     * @return factory for creating QueryBuilders. Might be null
>>>> +     */
>>>> +    public BatchQueryBuilderFactory getQueryBuilderFactory() {
>>>> +        return queryBuilderFactory;
>>>> +    }
>>>>
>>>> Cheers,
>>>> Andrus
>>>>
>>>> P.S. Maybe you can separately commit the @Deprecated part of the  
>>>> patch
>>>> ;-)
>>>>
>>>>
>>>>
>>>> On Jun 2, 2009, at 5:07 PM, Andrey Razumovsky wrote:
>>>>
>>>> For not to be unsubstantiated, I uploaded my vision of the  
>>>> feature. Note
>>>>
>>>>> that there is no modeler support. It allows to provide custom  
>>>>> factory
>>>>> that
>>>>> creates builders of INSERT, UPDATE, DELETE queries. 'Soft-delete'
>>>>> factory
>>>>> is
>>>>> included.
>>>>> I do not say that 'soft delete' checkbox is not needed, but  
>>>>> uploaded
>>>>> code
>>>>> is
>>>>> much more generic and allows to plug any behavior (maybe even  
>>>>> Ari's
>>>>> proposed
>>>>> 'versions'). 'Soft' strategy can be configured with 'deleted'  
>>>>> field name
>>>>> and
>>>>> will not fire UPDATE if such field does not exist in DB table.
>>>>>
>>>>> Please add your comments!
>>>>>
>>>>> Thanks,
>>>>> Andrey
>>>>>
>>>>> 2009/6/2 Andrus Adamchik <[hidden email]>
>>>>>
>>>>>
>>>>> On Jun 2, 2009, at 2:54 PM, Andrey Razumovsky wrote:
>>>>>>
>>>>>> But I'm no fan of adding some sort of 'soft' checkbox for  
>>>>>> dbattributes
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> I was suggesting marking entity with a "soft delete" checkbox  
>>>>>>> (not
>>>>>> individual attribute - this would make no sense), and creating a
>>>>>> criteria
>>>>>> based on qualifier that references an attribute.
>>>>>>
>>>>>> Modeler support will be covered by setting class name of strategy
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> I am afraid this approach will be rather arbitrary to the end  
>>>>>>> user,
>>>>>> so I
>>>>>> suggest we discuss it some more before putting it in Cayenne.  
>>>>>> Marking
>>>>>> an
>>>>>> entity to use "soft delete" based on some criteria is a clear and
>>>>>> understandable feature. Setting a "delete strategy" is not, and  
>>>>>> will
>>>>>> contribute to confusion. This is totally be ok as a backend  
>>>>>> extension
>>>>>> point,
>>>>>> but I will hate to see that as a general use feature.
>>>>>>
>>>>>> Andrus
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>
>>

[Andrus Adamchik]

On Jun 3, 2009, at 11:24 AM, Andrus Adamchik wrote:

>  You'd generate a BatchUpdateAction instead of delete or something  
> like that.

I mean a BatchUpdateQuery not Action.