Wsrtjtyk

I have another spring boot project that is able to populate datastore in a test case but I am just running into runtime issues with this project and I cannot see what I am doing differently :(

OS: XUbuntu 18.04.1

Java: 8 Updated 191

Spring boot: 2.1.0.RELEASE

GCP bom: 0.69.0-alpha

gcloud cloud-datastore-emulator: 224.0.0

The stacktrace is:

com.google.cloud.datastore.DatastoreException: I/O error

    at com.google.cloud.datastore.spi.v1.HttpDatastoreRpc.translate(HttpDatastoreRpc.java:129) ~[google-cloud-datastore-1.51.0.jar:1.51.0]

    at com.google.cloud.datastore.spi.v1.HttpDatastoreRpc.translate(HttpDatastoreRpc.java:114) ~[google-cloud-datastore-1.51.0.jar:1.51.0]

    at com.google.cloud.datastore.spi.v1.HttpDatastoreRpc.lookup(HttpDatastoreRpc.java:164) ~[google-cloud-datastore-1.51.0.jar:1.51.0]

    at com.google.cloud.datastore.DatastoreImpl$3.call(DatastoreImpl.java:384) ~[google-cloud-datastore-1.51.0.jar:1.51.0]

    at com.google.cloud.datastore.DatastoreImpl$3.call(DatastoreImpl.java:380) ~[google-cloud-datastore-1.51.0.jar:1.51.0]

    at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:105) ~[gax-1.34.0.jar:1.34.0]

    at com.google.cloud.RetryHelper.run(RetryHelper.java:74) ~[google-cloud-core-1.51.0.jar:1.51.0]

    at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:51) ~[google-cloud-core-1.51.0.jar:1.51.0]

    at com.google.cloud.datastore.DatastoreImpl.lookup(DatastoreImpl.java:379) ~[google-cloud-datastore-1.51.0.jar:1.51.0]

    at com.google.cloud.datastore.DatastoreImpl$ResultsIterator.loadResults(DatastoreImpl.java:355) ~[google-cloud-datastore-1.51.0.jar:1.51.0]

    at com.google.cloud.datastore.DatastoreImpl$ResultsIterator.<init>(DatastoreImpl.java:351) ~[google-cloud-datastore-1.51.0.jar:1.51.0]

    at com.google.cloud.datastore.DatastoreImpl.get(DatastoreImpl.java:341) ~[google-cloud-datastore-1.51.0.jar:1.51.0]

    at com.google.cloud.datastore.DatastoreImpl.get(DatastoreImpl.java:305) ~[google-cloud-datastore-1.51.0.jar:1.51.0]

    at com.google.cloud.datastore.DatastoreHelper.get(DatastoreHelper.java:46) ~[google-cloud-datastore-1.51.0.jar:1.51.0]

    at com.google.cloud.datastore.DatastoreImpl.get(DatastoreImpl.java:290) ~[google-cloud-datastore-1.51.0.jar:1.51.0]

    at com.mycompany.data.repo.impl.ConstantsRepositoryImpl.setProperty(ConstantsRepositoryImpl.java:51) ~[mycompany-data-repository-0.0.1-SNAPSHOT.jar:na]

    at com.mycompany.merchantrouting.MerchantRoutingApplicationTest.setup(MerchantRoutingApplicationTest.java:60) [test-classes/:na]

    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_191]

    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_191]

    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_191]

    at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_191]

    at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:515) [junit-platform-commons-1.3.1.jar:1.3.1]

    at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:115) [junit-jupiter-engine-5.3.1.jar:5.3.1]

    at org.junit.jupiter.engine.descriptor.ClassTestDescriptor.invokeMethodInExtensionContext(ClassTestDescriptor.java:436) [junit-jupiter-engine-5.3.1.jar:5.3.1]

    at org.junit.jupiter.engine.descriptor.ClassTestDescriptor.lambda$synthesizeBeforeEachMethodAdapter$14(ClassTestDescriptor.java:424) [junit-jupiter-engine-5.3.1.jar:5.3.1]

    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeBeforeEachMethods$2(TestMethodTestDescriptor.java:136) ~[junit-jupiter-engine-5.3.1.jar:5.3.1]

    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:72) ~[junit-platform-engine-1.3.1.jar:1.3.1]

    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeBeforeMethodsOrCallbacksUntilExceptionOccurs(TestMethodTestDescriptor.java:156) ~[junit-jupiter-engine-5.3.1.jar:5.3.1]

    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeBeforeEachMethods(TestMethodTestDescriptor.java:135) ~[junit-jupiter-engine-5.3.1.jar:5.3.1]

    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:110) ~[junit-jupiter-engine-5.3.1.jar:5.3.1]

    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:59) ~[junit-jupiter-engine-5.3.1.jar:5.3.1]

    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:105) ~[junit-platform-engine-1.3.1.jar:1.3.1]

    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:72) ~[junit-platform-engine-1.3.1.jar:1.3.1]

    at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:95) ~[junit-platform-engine-1.3.1.jar:1.3.1]

    at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:71) ~[junit-platform-engine-1.3.1.jar:1.3.1]

    at java.util.ArrayList.forEach(ArrayList.java:1257) ~[na:1.8.0_191]

    at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38) ~[junit-platform-engine-1.3.1.jar:1.3.1]

    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:110) ~[junit-platform-engine-1.3.1.jar:1.3.1]

    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:72) ~[junit-platform-engine-1.3.1.jar:1.3.1]

    at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:95) ~[junit-platform-engine-1.3.1.jar:1.3.1]

    at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:71) ~[junit-platform-engine-1.3.1.jar:1.3.1]

    at java.util.ArrayList.forEach(ArrayList.java:1257) ~[na:1.8.0_191]

    at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38) ~[junit-platform-engine-1.3.1.jar:1.3.1]

    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:110) ~[junit-platform-engine-1.3.1.jar:1.3.1]

    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:72) ~[junit-platform-engine-1.3.1.jar:1.3.1]

    at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:95) ~[junit-platform-engine-1.3.1.jar:1.3.1]

    at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:71) ~[junit-platform-engine-1.3.1.jar:1.3.1]

    at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32) ~[junit-platform-engine-1.3.1.jar:1.3.1]

    at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57) ~[junit-platform-engine-1.3.1.jar:1.3.1]

    at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51) ~[junit-platform-engine-1.3.1.jar:1.3.1]

    at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:170) ~[junit-platform-launcher-1.2.0.jar:1.2.0]

    at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:154) ~[junit-platform-launcher-1.2.0.jar:1.2.0]

    at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:90) ~[junit-platform-launcher-1.2.0.jar:1.2.0]

    at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:142) ~[surefire-junit-platform-2.22.0.jar:2.22.0]

    at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:117) ~[surefire-junit-platform-2.22.0.jar:2.22.0]

    at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:383) ~[surefire-booter-2.22.0.jar:2.22.0]

    at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:344) ~[surefire-booter-2.22.0.jar:2.22.0]

    at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125) ~[surefire-booter-2.22.0.jar:2.22.0]

    at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:417) ~[surefire-booter-2.22.0.jar:2.22.0]

Caused by: com.google.datastore.v1.client.DatastoreException: I/O error

    at com.google.datastore.v1.client.RemoteRpc.makeException(RemoteRpc.java:226) ~[datastore-v1-proto-client-1.6.0.jar:na]

    at com.google.datastore.v1.client.RemoteRpc.call(RemoteRpc.java:195) ~[datastore-v1-proto-client-1.6.0.jar:na]

    at com.google.datastore.v1.client.Datastore.lookup(Datastore.java:95) ~[datastore-v1-proto-client-1.6.0.jar:na]

    at com.google.cloud.datastore.spi.v1.HttpDatastoreRpc.lookup(HttpDatastoreRpc.java:162) ~[google-cloud-datastore-1.51.0.jar:1.51.0]

    ... 56 common frames omitted

Caused by: java.net.ConnectException: Connection refused (Connection refused)

    at java.net.PlainSocketImpl.socketConnect(Native Method) ~[na:1.8.0_191]

    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) ~[na:1.8.0_191]

    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) ~[na:1.8.0_191]

    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) ~[na:1.8.0_191]

    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[na:1.8.0_191]

    at java.net.Socket.connect(Socket.java:589) ~[na:1.8.0_191]

    at sun.net.NetworkClient.doConnect(NetworkClient.java:175) ~[na:1.8.0_191]

    at sun.net.www.http.HttpClient.openServer(HttpClient.java:463) ~[na:1.8.0_191]

    at sun.net.www.http.HttpClient.openServer(HttpClient.java:558) ~[na:1.8.0_191]

    at sun.net.www.http.HttpClient.<init>(HttpClient.java:242) ~[na:1.8.0_191]

    at sun.net.www.http.HttpClient.New(HttpClient.java:339) ~[na:1.8.0_191]

    at sun.net.www.http.HttpClient.New(HttpClient.java:357) ~[na:1.8.0_191]

    at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1220) ~[na:1.8.0_191]

    at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1156) ~[na:1.8.0_191]

    at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1050) ~[na:1.8.0_191]

    at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:984) ~[na:1.8.0_191]

    at sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1334) ~[na:1.8.0_191]

    at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1309) ~[na:1.8.0_191]

    at com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:77) ~[google-http-client-1.24.1.jar:1.24.1]

    at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:981) ~[google-http-client-1.24.1.jar:1.24.1]

    at com.google.datastore.v1.client.RemoteRpc.call(RemoteRpc.java:183) ~[datastore-v1-proto-client-1.6.0.jar:na]

    ... 58 common frames omitted

I have the following test configuration for the Datastore:

package com.mycompany.merchantrouting;



import com.mycompany.data.repo.DatastoreRegistry;

import com.google.cloud.datastore.Datastore;

import com.google.cloud.datastore.testing.LocalDatastoreHelper;

import org.springframework.beans.factory.annotation.Value;

import org.springframework.context.annotation.Bean;

import org.springframework.context.annotation.Configuration;

import org.springframework.context.annotation.Profile;



@Configuration

@Profile("test")

public class TestConfiguration {



    @Bean(destroyMethod = "stop")

    public LocalDatastoreHelper dsHelper() throws Exception {

        final LocalDatastoreHelper helper = LocalDatastoreHelper.create();

        helper.start();



        return helper;

    }



    @Bean

    public DatastoreRegistry testDatastore(@Value("${mycompany.namespace}") final String namespace,

            final LocalDatastoreHelper helper) {

        final Datastore ds = helper.getOptions(namespace).getService();



        return (s) -> ds;

    }

}

When calling Datastore.add or Datastore.put I see the stacktrace. Again I cannot see anything obvious that is different. I am populating the same entity 'Kinds'. The port appears to be randomly assigned in the BaseEmulatorHelper so it does not appear to be a port conflict

Edit 2018-11-08

I've turned on trace logging. It appears that there is a request and response is being sent.

09:40:00.506 [main] DEBUG c.g.datastore.v1.client.RemoteRpc - remote datastore call lookup

09:40:00.507 [main] INFO  c.g.api.client.http.HttpTransport - -------------- REQUEST  --------------

POST http://localhost:43639/v1/projects/test-project-3de2413c-dceb-4506-8121-aeaa75e97486:lookup

Accept-Encoding: gzip

User-Agent: gcloud-java/1.49.0 Google-HTTP-Java-Client/1.24.1 (gzip)

x-goog-api-format-version: 2

x-goog-api-client: gl-java/1.8.0_191 gccl/1.49.0 gax/1.32.0

Content-Type: application/x-protobuf

Content-Length: 70



09:40:00.507 [main] INFO  c.g.api.client.http.HttpTransport - curl -v --compressed -X POST -H 'Accept-Encoding: gzip' -H 'User-Agent: gcloud-java/1.49.0 Google-HTTP-Java-Client/1.24.1 (gzip)' -H 'x-goog-api-format-version: 2' -H 'x-goog-api-client: gl-java/1.8.0_191 gccl

/1.49.0 gax/1.32.0' -H 'Content-Type: application/x-protobuf' -d '@-' -- 'http://localhost:43639/v1/projects/test-project-3de2413c-dceb-4506-8121-aeaa75e97486:lookup' << $$$

09:40:00.507 [main] TRACE s.n.w.p.http.HttpURLConnection - ProxySelector Request for http://localhost:43639/v1/projects/test-project-3de2413c-dceb-4506-8121-aeaa75e97486:lookup

09:40:00.508 [main] TRACE s.n.w.p.http.HttpURLConnection - HttpClient.available(): SocketTimeout: its available

09:40:00.508 [main] TRACE s.n.w.p.http.HttpURLConnection - KeepAlive stream retrieved from the cache, sun.net.www.http.HttpClient(http://localhost:43639/v1/projects/test-project-3de2413c-dceb-4506-8121-aeaa75e97486:lookup)

09:40:00.508 [main] TRACE s.n.w.p.http.HttpURLConnection - Proxy used: DIRECT

09:40:00.508 [main] DEBUG s.n.w.p.http.HttpURLConnection - sun.net.www.MessageHeader@3f1ed06810 pairs: {POST /v1/projects/test-project-3de2413c-dceb-4506-8121-aeaa75e97486:lookup HTTP/1.1: null}{Accept-Encoding: gzip}{User-Agent: gcloud-java/1.49.0 Google-HTTP-Java-Clien

t/1.24.1 (gzip)}{x-goog-api-format-version: 2}{x-goog-api-client: gl-java/1.8.0_191 gccl/1.49.0 gax/1.32.0}{Content-Type: application/x-protobuf}{Host: localhost:43639}{Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2}{Connection: keep-alive}{Content-Length: 

70}

09:40:00.522 [main] INFO  c.g.api.client.http.HttpTransport - Total: 70 bytes

09:40:00.523 [main] INFO  c.g.api.client.http.HttpTransport -  D

3 1test-project-3de2413c-dceb-4506-8121-aeaa75e97486 

 Constants  

09:40:00.524 [main] TRACE s.n.w.p.http.HttpURLConnection - KeepAlive stream used: http://localhost:43639/v1/projects/test-project-3de2413c-dceb-4506-8121-aeaa75e97486:lookup

09:40:00.524 [main] DEBUG s.n.w.p.http.HttpURLConnection - sun.net.www.MessageHeader@753aca853 pairs: {null: HTTP/1.1 200 OK}{content-type: application/x-protobuf}{content-length: 76}

09:40:00.525 [main] INFO  c.g.api.client.http.HttpTransport - -------------- RESPONSE --------------

HTTP/1.1 200 OK

content-length: 76

content-type: application/x-protobuf



09:40:00.526 [main] DEBUG c.g.datastore.v1.client.RemoteRpc - remote datastore call lookup took 20 ms

09:40:00.529 [main] INFO  c.g.api.client.http.HttpTransport - Total: 76 bytes

09:40:00.532 [main] INFO  c.g.api.client.http.HttpTransport -  J

F

D

3 1test-project-3de2413c-dceb-4506-8121-aeaa75e97486 

 Constants    

Seems like the same issue as Dataflow + Datastore = DatastoreException: I/O error ?
I assume you're using the open sourced google-cloud-java library? Yeah the LocalDatastoreHelper is a bit confusing, but that's only for testing purpose. Per README, the recommended way to construct a Datastore object is:

import com.google.cloud.datastore.Datastore;

import com.google.cloud.datastore.DatastoreOptions;

Datastore datastore = DatastoreOptions.getDefaultInstance().getService();

Then you can do:

datastore.put(stuff);

Please reply back if there's still issue.