MappingCouchbaseConverter.read() is not recursive #1312

JesusTheHun · 2022-01-28T19:19:29Z

Using Cluster.query() I'm parsing the result into classes that have child object.
A quick exemple to visualise :

@Data
public class A {
    String someProp;
    SubA nestedObject;

    @Data
    public static SubA() {
        String someNestedProp;
    }
}

Doing converter.read(A.class, document) leads to nestedObject being a HashMap<String, String> instead of a SubA as anyone would expect. Am I doing this wrong or are my expectations correct ?

The text was updated successfully, but these errors were encountered:

mikereiche · 2022-02-01T00:22:05Z

I made an ARepsitory and did a save followed by a find.

git clone [email protected]:spring-projects/spring-data-couchbase.git

package org.springframework.data.couchbase.domain;

import lombok.AllArgsConstructor;
import lombok.Data;
import org.springframework.data.annotation.Id;
import org.springframework.data.couchbase.core.mapping.Document;

@Document
@Data
@AllArgsConstructor
public class A {
  @Id
  String someProp;
  SubA nestedObject;

  public String getId() {
    return someProp;
  }

  public String toString(){
    return "A{ someProp="+someProp+" nestedObject="+nestedObject+"}";
  }

  @Data
  @AllArgsConstructor
  public static class SubA {
    String someNestedProp;
    public String toString(){
      return "SubA{ someNestedProp="+someNestedProp+"}";
    }
  }

}

package org.springframework.data.couchbase.repository;

import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.couchbase.config.AbstractCouchbaseConfiguration;
import org.springframework.data.couchbase.domain.A;
import org.springframework.data.couchbase.domain.ARepository;
import org.springframework.data.couchbase.repository.config.EnableCouchbaseRepositories;
import org.springframework.data.couchbase.util.ClusterAwareIntegrationTests;
import org.springframework.data.couchbase.util.ClusterType;
import org.springframework.data.couchbase.util.IgnoreWhen;
import org.springframework.test.context.junit.jupiter.SpringJUnitConfig;

import java.lang.reflect.InvocationTargetException;
import java.util.Optional;

import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertTrue;

/**
 * Repository KV tests
 *
 * @author Michael Nitschinger
 * @author Michael Reiche
 */
@SpringJUnitConfig(CouchbaseARepositoryKeyValueIntegrationTests.Config.class)
@IgnoreWhen(clusterTypes = ClusterType.MOCKED)
public class CouchbaseARepositoryKeyValueIntegrationTests extends ClusterAwareIntegrationTests {

  @Autowired
  ARepository aRepository;

  @Test
  @IgnoreWhen(clusterTypes = ClusterType.MOCKED)
  void saveAndFindById() throws NoSuchMethodException, InvocationTargetException, IllegalAccessException {
    A.SubA subA = new A.SubA("this is someNestedProp");
    A a = new A("this is someProp", subA);
    a = aRepository.save(a);
    Optional<A> found = aRepository.findById(a.getId());
    assertTrue(found.isPresent());
    assertEquals(a,
        found.get());
    System.out.println("a="+a);
    System.out.println("a.suba="+a.getSomeProp());
    aRepository.delete(a);
  }


  @Configuration
  @EnableCouchbaseRepositories("org.springframework.data.couchbase")
  static class Config extends AbstractCouchbaseConfiguration {

    @Override
    public String getConnectionString() {
      return connectionString();
    }

    @Override
    public String getUserName() {
      return config().adminUsername();
    }

    @Override
    public String getPassword() {
      return config().adminPassword();
    }

    @Override
    public String getBucketName() {
      return bucketName();
    }

  }

}

src/test/resources/integration.properties

# Couchbase Integration-Test Default Properties
# If set to false, it is assumed that the host is managing the cluster and
# as a result no containers or anything will be spun up.
# Options: containerized, mocked, unmanaged
cluster.type=unmanaged <<<< change to unmanaged
# Default configs for both cases
cluster.adminUsername=Administrator
cluster.adminPassword=password
# Default configs for the mocked environment
cluster.mocked.numNodes=1
cluster.mocked.numReplicas=1
# Entry point configuration if not managed
# value of hostname and ns_server port
cluster.unmanaged.seed=10.144.220.101:8091 <<<< my server
cluster.unmanaged.numReplicas=0

I get the expected result:

a=A{ someProp=this is someProp nestedObject=SubA{ someNestedProp=this is someNestedProp}}
a.suba=this is someProp

From CouchbaseTemplate.support().decodeEntity()

T readEntity = converter.read(entityClass, (CouchbaseDocument) translationService.decode(source, converted));

mikereiche · 2022-02-01T00:50:13Z

the stacktrace from decodeEntity() will be this. See the two calls at read:243, MappingCouchbaseConverter? That's the recursion. The second call is initiated from read:Value:846 because the 'value' is a(nother) CouchbaseDocument.

readValue:845, MappingCouchbaseConverter (org.springframework.data.couchbase.core.convert) [2]
access$300:85, MappingCouchbaseConverter (org.springframework.data.couchbase.core.convert)
getPropertyValue:964, MappingCouchbaseConverter$CouchbasePropertyValueProvider (org.springframework.data.couchbase.core.convert)
getValueInternal:308, MappingCouchbaseConverter (org.springframework.data.couchbase.core.convert)
doWithPersistentProperty:276, MappingCouchbaseConverter$1 (org.springframework.data.couchbase.core.convert)
doWithPersistentProperty:268, MappingCouchbaseConverter$1 (org.springframework.data.couchbase.core.convert)
doWithProperties:360, BasicPersistentEntity (org.springframework.data.mapping.model)
read:268, MappingCouchbaseConverter (org.springframework.data.couchbase.core.convert)
read:243, MappingCouchbaseConverter (org.springframework.data.couchbase.core.convert)
readValue:846, MappingCouchbaseConverter (org.springframework.data.couchbase.core.convert) [1]
access$300:85, MappingCouchbaseConverter (org.springframework.data.couchbase.core.convert)
getPropertyValue:964, MappingCouchbaseConverter$CouchbasePropertyValueProvider (org.springframework.data.couchbase.core.convert)
getPropertyValue:913, MappingCouchbaseConverter$CouchbasePropertyValueProvider (org.springframework.data.couchbase.core.convert)
getParameterValue:74, PersistentEntityParameterValueProvider (org.springframework.data.mapping.model)
getParameterValue:53, SpELExpressionParameterValueProvider (org.springframework.data.mapping.model)
extractInvocationArguments:276, ClassGeneratingEntityInstantiator (org.springframework.data.mapping.model)
createInstance:248, ClassGeneratingEntityInstantiator$EntityInstantiatorAdapter (org.springframework.data.mapping.model)
createInstance:89, ClassGeneratingEntityInstantiator (org.springframework.data.mapping.model)
read:265, MappingCouchbaseConverter (org.springframework.data.couchbase.core.convert)
read:243, MappingCouchbaseConverter (org.springframework.data.couchbase.core.convert)
read:200, MappingCouchbaseConverter (org.springframework.data.couchbase.core.convert)
read:85, MappingCouchbaseConverter (org.springframework.data.couchbase.core.convert)
decodeEntity:106, CouchbaseTemplateSupport (org.springframework.data.couchbase.core)

btw - The CouchbaseTemplateSupport object and its decodeEntity() method are public, so you shouldn't have to dig into MappingCouchbaseConverter.read(). #1210

mikereiche · 2022-02-01T00:58:03Z

If you post your code, I'll run it in the debugger to see what's going on.

JesusTheHun · 2022-02-01T13:22:56Z

Here is the code :

Class definition

@Data
abstract public class BaseManageableEntity {
    @Id
    protected String id;

    @Field("dag_paths")
    protected List<DagPath> dagPaths = new ArrayList<>();

    abstract public ManageableType getManageableType();

    @Data
    @AllArgsConstructor
    @EqualsAndHashCode
    public static class DagPath {
        private LinkType pathType;
        private List<DagNode> path;

        @Data
        @EqualsAndHashCode
        public static class DagNode {
            private String id;
            private ManageableType type;

            private DagNode() {}

            public static DagNode of(BaseManageableEntity manageable) {
                if (manageable.getId() == null) throw new IllegalArgumentException();
                if (manageable.getManageableType() == null) throw new IllegalArgumentException();

                var node = new DagNode();
                node.id = manageable.getId();
                node.type = manageable.getManageableType();

                return node;
            }
        }
    }
}

@Document
@Data
@EqualsAndHashCode(callSuper = true)
public class Foo extends BaseManageableEntity {
    String title;
    public ManageableType getManageableType() {
        return ManageableType. ManageableTypeOne;
    }
}

public enum ManageableType {
    ManageableTypeOne, ManageableTypeTwo
}

public enum LinkType {
    LinkTypeOne, LinkTypeTwo
}

// Utility functions
public <T> T toDomainEntity(byte[] obj, Class<T> clazz) {
    JsonObject json = serializer.deserialize(JsonObject.class, obj);
    Class<?> targetClass = Class.forName(json.get("_class").toString());

    if (!clazz.isAssignableFrom(targetClass)) {
        throw new IllegalArgumentException(String.format("Trying to assign %s to %s", targetClass, clazz));
    }

    CouchbaseDocument document = new CouchbaseDocument()
            .setId(json.get(TemplateUtils.SELECT_ID).toString())
            .setContent(json);
    return (T) couchbaseTemplate.getConverter().read(targetClass, document);
}

public <T> List<T> toDomainEntities(List<byte[]> rows, Class<T> clazz) {
    return rows.stream().map(obj -> toDomainEntity(obj, clazz)).collect(Collectors.toList());
}

// Execution
QueryResult results = cluster.query(localQuery, QueryOptions.queryOptions().parameters(JsonObject.from(localMap)).adhoc(isAdHoc()));
return toDomainEntities(results.rowsAs(byte[].class), clazz);

mikereiche · 2022-02-01T15:24:03Z

Could you provide an input data? Thanks.

mikereiche · 2022-02-01T15:25:39Z

The difference could be that the fields are lists of sub-documents. But it should still work

JesusTheHun · 2022-02-01T16:03:34Z

I have edited the code in my previous comment to be more complete.
Very basic stuff for data input:

var a = new Foo();
a.setId("a");

var b = new Foo();
b.setId("b");

var dagPath = new BaseManageableEntity.DagPath(LinkType.LinkTypeOne, List.of(BaseManageableEntity.DagPath.DagNode.of(a), BaseManageableEntity.DagPath.DagNode.of(b)));
b.getDagPaths().add(dagPath);

mikereiche · 2022-02-01T16:58:07Z

Foo doesn't implement getManageableType() from BaseManageableEntity.

JesusTheHun · 2022-02-01T17:12:25Z

I've updated the code.

mikereiche · 2022-02-01T17:23:42Z

couchbaseTemplate.insertById()/findById() (which uses CouchbaseTemplateSupport.decodeEntity) seems to work:

  @Test
  @IgnoreWhen(clusterTypes = ClusterType.MOCKED)
  void saveAndFindById2() {

    Foo a = new Foo();
    a.setId("a");

    var b = new Foo();
    b.setId("b");

    BaseManageableEntity.DagPath dagPath = new BaseManageableEntity.DagPath(LinkType.LinkTypeOne,
        List.of(BaseManageableEntity.DagPath.DagNode.of(a),
            BaseManageableEntity.DagPath.DagNode.of(b)));
    b.getDagPaths()
        .add(dagPath);

    try {
      couchbaseTemplate.removeById(Foo.class).one(b.getId());
    } catch(Exception e){
      //System.err.println(e);
    }
    couchbaseTemplate.insertById(Foo.class).one(b);

    var c = couchbaseTemplate.findById(Foo.class).one(b.getId());
    System.err.println("Foo: " +c);
    System.err.println("DagPath: "+c.getDagPaths());

    couchbaseTemplate.removeById(Foo.class).one(b.getId());
  }

Foo: Foo(title=null, mt=ManageableTypeOne)
DagPath: [BaseManageableEntity.DagPath(pathType=LinkTypeOne, path=[BaseManageableEntity.DagPath.DagNode(id=a, type=ManageableTypeOne), BaseManageableEntity.DagPath.DagNode(id=b, type=ManageableTypeOne)])]

Where does 'serializer' in toDomainEntity come from?

JesusTheHun · 2022-02-01T17:27:15Z

Where does 'serializer' in toDomainEntity come from?

It comes from cluster.environment().jsonSerializer().

JesusTheHun · 2022-02-01T17:29:57Z

I tried to use CouchbaseTemplateSupport but for that I needed to upgrade to 4.3.1 and then I met the breaking change exposed in #1315

mikereiche · 2022-02-01T18:16:32Z

The issue here (in 1312) is that in the CouchbaseDocument Foo, the List<> dagPath is an ArrayList (maybe it should be a CouchbaseList?) and it falls through to the else. I'll look into it.

In the meantime, can you use either the template or repository? Or maybe we can skip the CouchbaseDocument and go from use a String as the source.

	private <R> R readValue(Object value, TypeInformation<?> type, Object parent) {
		Class<?> rawType = type.getType();

		if (conversions.hasCustomReadTarget(value.getClass(), rawType)) {
			return (R) conversionService.convert(value, rawType);
		} else if (value instanceof CouchbaseDocument) {
			return (R) read(type, (CouchbaseDocument) value, parent);
		} else if (value instanceof CouchbaseList) {
			return (R) readCollection(type, (CouchbaseList) value, parent);
		} else {
			return (R) getPotentiallyConvertedSimpleRead(value, rawType);
		}
	}

mikereiche · 2022-02-01T19:13:00Z

Replace your toEntity() with this:

  public <T> T toDomainEntity(byte[] obj, Class<T> clazz) throws RuntimeException {
    JsonObject json = serializer.deserialize(JsonObject.class,
        obj);
    Class<?> targetClass = null;
    String id = null;
    long cas = 0;
    try {
      targetClass = Class.forName(json.get("_class")
          .toString());
      id = json.getString("__id");
      cas = json.getLong("__cas");
    } catch (ClassNotFoundException e) {
      throw new RuntimeException(e);
    }

    if (!clazz.isAssignableFrom(targetClass)) {
      throw new IllegalArgumentException(String.format("Trying to assign %s to %s",
          targetClass,
          clazz));
    }

    return (T) couchbaseTemplate.support()
        .decodeEntity(id,
            new String(obj),
            cas,
            targetClass);
 }

bme: Foo{title: null, mt: ManageableTypeOne, dagPaths: [DagPath{DagNodes : [DagNode{id : a}, DagNode{id : b}]}]}

mikereiche · 2022-02-02T22:10:35Z

This looks to be the same issue as #1276

mikereiche · 2022-02-03T18:44:30Z

The row needs to be decoded by the translationService. The translationService can be @Autowired from the config (this is what CouchbaseTemplateSupport.decodeEntity() does. The ArrayList is replaced by a CouchbaseList of CouchbaseDocuments, which MappingCouchbaseConverter converts to a List.

return  (T)couchbaseTemplate.getConverter()
    .read(targetClass,
        (CouchbaseDocument)translationService.decode(  new String (obj), document));

JesusTheHun · 2022-02-10T17:37:27Z

After moving to 4.3.2-SNAPSHOT and using to the translation service, this issue is resolved ! Thank you Michael !

mikereiche · 2022-02-10T18:03:20Z

I'm going to keep this open as someone else hit this and I need to update the documentation.

JesusTheHun · 2022-02-11T09:59:14Z

I'm going to keep this open as someone else hit this and I need to update the documentation.

Is the documentation open to PRs ?

mikereiche · 2022-02-11T15:46:08Z

Yes. They live in https://github.com/spring-projects/spring-data-couchbase/tree/main/src/main/asciidoc
btw - there is a PR for querydsl at #1330
I don't seem to be able to add you as a reviewer.

JesusTheHun · 2022-02-11T17:04:11Z

btw - there is a PR for querydsl at #1330
I don't seem to be able to add you as a reviewer.

Having a sample project would help a lot to understand the architecture.
Also a draft of documentation to get the gist of the current capabilities

mikereiche · 2022-02-11T17:23:38Z

There are samples in src/test/java/org/springframework/data/couchbase/repository/query/CouchbaseRepositoryQuerydslIntegrationTests.java
They run with

changing src/test/resources/integration.properties
cluster.type=unmanaged
cluster.unmanaged.seed=10.144.220.101:8091
mvn integration-test -Dit.test=CouchbaseRepositoryQuerydslIntegrationTests

Bonus - to only create the bucket once and not delete it at the end of the tests.

src/test/java/org/springframework/data/couchbase/util/UnmanagedTestCluster.java
63 bucketname = "my_bucket"; // UUID.randomUUID().toString();

JesusTheHun changed the title ~~MappingCouchbaseConverter.read() should be recursive~~ MappingCouchbaseConverter.read() is not recursive Jan 28, 2022

spring-projects-issues added the status: waiting-for-triage An issue we've not yet triaged label Jan 28, 2022

mikereiche added status: duplicate A duplicate of another issue type: bug A general bug and removed status: waiting-for-triage An issue we've not yet triaged labels Feb 3, 2022

mikereiche mentioned this issue Feb 3, 2022

Invalid class type on nested object - MappingCouchbaseConverter - couchbase transactions #1276

Closed

mikereiche added type: documentation A documentation update and removed status: duplicate A duplicate of another issue labels Feb 3, 2022

JesusTheHun closed this as completed Feb 10, 2022

mikereiche reopened this Feb 10, 2022

mikereiche removed the type: bug A general bug label Jun 28, 2022

mikereiche closed this as completed May 10, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

MappingCouchbaseConverter.read() is not recursive #1312

MappingCouchbaseConverter.read() is not recursive #1312

JesusTheHun commented Jan 28, 2022 •

edited

Loading

mikereiche commented Feb 1, 2022

mikereiche commented Feb 1, 2022 •

edited

Loading

mikereiche commented Feb 1, 2022

JesusTheHun commented Feb 1, 2022 •

edited

Loading

mikereiche commented Feb 1, 2022

mikereiche commented Feb 1, 2022

JesusTheHun commented Feb 1, 2022 •

edited

Loading

mikereiche commented Feb 1, 2022 •

edited

Loading

JesusTheHun commented Feb 1, 2022

mikereiche commented Feb 1, 2022 •

edited

Loading

JesusTheHun commented Feb 1, 2022 •

edited

Loading

JesusTheHun commented Feb 1, 2022 •

edited

Loading

mikereiche commented Feb 1, 2022

mikereiche commented Feb 1, 2022

mikereiche commented Feb 2, 2022

mikereiche commented Feb 3, 2022 •

edited

Loading

JesusTheHun commented Feb 10, 2022

mikereiche commented Feb 10, 2022

JesusTheHun commented Feb 11, 2022

mikereiche commented Feb 11, 2022

JesusTheHun commented Feb 11, 2022

mikereiche commented Feb 11, 2022

MappingCouchbaseConverter.read() is not recursive #1312

MappingCouchbaseConverter.read() is not recursive #1312

Comments

JesusTheHun commented Jan 28, 2022 • edited Loading

mikereiche commented Feb 1, 2022

mikereiche commented Feb 1, 2022 • edited Loading

mikereiche commented Feb 1, 2022

JesusTheHun commented Feb 1, 2022 • edited Loading

mikereiche commented Feb 1, 2022

mikereiche commented Feb 1, 2022

JesusTheHun commented Feb 1, 2022 • edited Loading

mikereiche commented Feb 1, 2022 • edited Loading

JesusTheHun commented Feb 1, 2022

mikereiche commented Feb 1, 2022 • edited Loading

JesusTheHun commented Feb 1, 2022 • edited Loading

JesusTheHun commented Feb 1, 2022 • edited Loading

mikereiche commented Feb 1, 2022

mikereiche commented Feb 1, 2022

mikereiche commented Feb 2, 2022

mikereiche commented Feb 3, 2022 • edited Loading

JesusTheHun commented Feb 10, 2022

mikereiche commented Feb 10, 2022

JesusTheHun commented Feb 11, 2022

mikereiche commented Feb 11, 2022

JesusTheHun commented Feb 11, 2022

mikereiche commented Feb 11, 2022

JesusTheHun commented Jan 28, 2022 •

edited

Loading

mikereiche commented Feb 1, 2022 •

edited

Loading

JesusTheHun commented Feb 1, 2022 •

edited

Loading

JesusTheHun commented Feb 1, 2022 •

edited

Loading

mikereiche commented Feb 1, 2022 •

edited

Loading

mikereiche commented Feb 1, 2022 •

edited

Loading

JesusTheHun commented Feb 1, 2022 •

edited

Loading

JesusTheHun commented Feb 1, 2022 •

edited

Loading

mikereiche commented Feb 3, 2022 •

edited

Loading