Squashed commit of the following:

commit b3d92d0
Author: Alex Rosengarten <[email protected]>
Date:   Wed Mar 4 12:17:17 2020 -0800

    updated test name

commit ba7884b
Author: Alex Rosengarten <[email protected]>
Date:   Wed Mar 4 12:16:09 2020 -0800

    nested unit tests

commit ef36fdf
Author: Alex Rosengarten <[email protected]>
Date:   Wed Mar 4 12:15:00 2020 -0800

    renamed to tryResolve

commit 5703f02
Author: Alex Rosengarten <[email protected]>
Date:   Wed Mar 4 12:07:18 2020 -0800

    rm generator

commit 658cb1a
Author: Alex Rosengarten <[email protected]>
Date:   Wed Mar 4 12:01:54 2020 -0800

    implemented more review suggestions

commit 0056ee3
Merge: cae89c3 0f0f82a
Author: Alex Rosengarten <[email protected]>
Date:   Wed Mar 4 11:54:23 2020 -0800

    Merge branch 'master' of github.com:PolymerLabs/arcs into r2p

commit 0f0f82a
Author: Maria Mandlis <[email protected]>
Date:   Wed Mar 4 09:33:29 2020 -0800

    add queryable capability (PolymerLabs#4794)

commit d032c75
Author: Maria Mandlis <[email protected]>
Date:   Wed Mar 4 09:23:07 2020 -0800

    add creatimeTimestamp to entities (kt) (PolymerLabs#4823)

commit 30267fc
Author: jblebrun <[email protected]>
Date:   Wed Mar 4 09:09:39 2020 -0800

    Remove some unused deps (PolymerLabs#4831)

commit c519181
Author: Cameron Silvestrini <[email protected]>
Date:   Wed Mar 4 17:14:07 2020 +1100

    Increase size of DatabaseImplTest to medium (PolymerLabs#4827)

    * Increase size of DatabaseImplTest to medium

    Was a bit flaky.

    * Check flakiness

    * Revert presubmit tweak

commit fa621c6
Author: jblebrun <[email protected]>
Date:   Tue Mar 3 22:05:07 2020 -0800

    Fix a race condition when setting up `ServiceStore` message channel (PolymerLabs#4826)

    This seems to be the cause of the flakiness in `AndroidAllocatorTest`
    and `AndroidHandleManagerTest` (PolymerLabs#4781)

commit 78bf4f0
Author: Gogul Balakrishnan <[email protected]>
Date:   Tue Mar 3 18:00:49 2020 -0800

    Utility to convert type proto to field type (if possible). (PolymerLabs#4814)

commit cae89c3
Merge: 0360752 e587539
Author: Alex Rosengarten <[email protected]>
Date:   Tue Mar 3 15:50:34 2020 -0800

    Merge branch 'master' of github.com:PolymerLabs/arcs into r2p

commit e587539
Author: jblebrun <[email protected]>
Date:   Tue Mar 3 15:07:16 2020 -0800

    Apply simplified test pattern tests in `arcs.android.host` (PolymerLabs#4821)

    As in 5c6310d

commit 1af1aac
Author: Joshua Pratt <[email protected]>
Date:   Wed Mar 4 09:48:15 2020 +1100

    Multinomials (PolymerLabs#4804)

    * multinomial

    * added tests

    * making constant readable again

    * reindent

    * added multivariate rearrangement tests

    * updated comment

    * pr comments:

    Co-authored-by: Ragav Sachdeva <[email protected]>

commit 0360752
Author: Alex Rosengarten <[email protected]>
Date:   Tue Mar 3 11:42:31 2020 -0800

    fix, bad comparison op

commit 4b05a28
Merge: f4c1d26 99000c2
Author: Alex Rosengarten <[email protected]>
Date:   Tue Mar 3 11:18:14 2020 -0800

    Merge branch 'r2p' of github.com:alxrsngrtn/arcs into r2p

commit f4c1d26
Merge: 140111f 5c6310d
Author: Alex Rosengarten <[email protected]>
Date:   Tue Mar 3 11:14:35 2020 -0800

    Merge branch 'master' of github.com:PolymerLabs/arcs into r2p
    Fixed lint errors

commit 99000c2
Merge: 140111f 5c6310d
Author: Alex Rosengarten <[email protected]>
Date:   Tue Mar 3 11:14:35 2020 -0800

    Merge branch 'master' of github.com:PolymerLabs/arcs into r2p
    Fixed lint errors

commit 140111f
Author: Alex Rosengarten <[email protected]>
Date:   Tue Mar 3 11:08:12 2020 -0800

    Removed flatMap, added TODOs and link to GH issue

commit 5c6310d
Author: jblebrun <[email protected]>
Date:   Tue Mar 3 11:02:34 2020 -0800

    Simplify service test pattern (PolymerLabs#4817)

    * Make a simple test lifecycle registry, instead of creating empty
    testactivity

    * Remove use of `runBlockingTest`: according to
    Kotlin/kotlinx.coroutines#1222, if the test
    results in coroutines being finished on other threads, it's possible to
    receive "This job has not yet completed" exceptions, even though the
    jobs were properly terminated. Since we don't need the delay-skipping
    properties of `runBlockingTest`, I think it's OK to use `runBlocking`.

commit 472bc84
Author: Alex Rosengarten <[email protected]>
Date:   Tue Mar 3 10:27:56 2020 -0800

    Improved build rules

commit ca1ebf8
Author: Alex Rosengarten <[email protected]>
Date:   Tue Mar 3 10:17:13 2020 -0800

    Impl suggestsions for r2p

commit b5578ea
Author: Jason Feinstein <[email protected]>
Date:   Tue Mar 3 09:54:17 2020 -0800

    Add tests for dereferencing references to the HandleManagerTest(s) (PolymerLabs#4816)

    * Add dereferencing tests to the android handle manager test.

    * Add dereferencing tests to core HandleManager.

commit a695797
Author: Jason Feinstein <[email protected]>
Date:   Mon Mar 2 18:14:07 2020 -0800

    Create RawEntityDereferencerTest, storage Reference-> CrdtEntity.Reference (PolymerLabs#4812)

    * Create RawEntityDereferencerTest, make storage Reference implement CrdtEntity.Reference.

    Also: Create ParcelableReference.

    * Add dep. Also, apparently read/writeBoolean is Q-only.

    * Just write null if there is no version map.

    * Just write null if there is no version map.

commit ba7a107
Author: Gogul Balakrishnan <[email protected]>
Date:   Mon Mar 2 17:51:35 2020 -0800

    Add a decoder for PrimitiveTypeProto and an option to disable android constraints in BUILD. (PolymerLabs#4793)

commit f6df54a
Author: Alex Rosengarten <[email protected]>
Date:   Mon Mar 2 15:44:32 2020 -0800

    tools/sigh lint

commit bede8d9
Author: Alex Rosengarten <[email protected]>
Date:   Mon Mar 2 14:52:02 2020 -0800

    Fixed build rule, simplified runtime

commit 62dcc57
Author: Alex Rosengarten <[email protected]>
Date:   Mon Mar 2 14:38:14 2020 -0800

    Fixed test, added TODO

Author: alxmrs

java/arcs/android/crdt/ParcelableRawEntity.kt

@@ -39,6 +39,7 @@ data class ParcelableRawEntity(
                 parcel.writeTypedObject(ParcelableReferencable(it), flags)
             }
         }
+        parcel.writeLong(actual.creationTimestamp)
         parcel.writeLong(actual.expirationTimestamp)
     }
 
@@ -67,10 +68,19 @@ data class ParcelableRawEntity(
                 collections[key] = set
             }
 
+            @Suppress("GoodTime") // use Instant
+            val creationTimestamp = requireNotNull(parcel.readLong())
+
             @Suppress("GoodTime") // use Instant
             val expirationTimestamp = requireNotNull(parcel.readLong())
 
-            val rawEntity = RawEntity(id, singletons, collections, expirationTimestamp)
+            val rawEntity = RawEntity(
+                id,
+                singletons,
+                collections,
+                creationTimestamp,
+                expirationTimestamp
+            )
             return ParcelableRawEntity(rawEntity)
         }
 

java/arcs/core/common/Referencable.kt

@@ -19,6 +19,11 @@ interface Referencable {
     /** Unique identifier of the Referencable object. */
     val id: ReferenceId
 
+    /** Creation timestamp (in millis) on the Referencable object. */
+    var creationTimestamp: Long
+        get() = TODO("not implemented")
+        set(@Suppress("UNUSED_PARAMETER") value) = TODO("not implemented")
+
     /** Expiration timestamp (in millis) on the Referencable object. */
     var expirationTimestamp: Long
         get() = TODO("not implemented")

java/arcs/core/data/RawEntity.kt

@@ -29,21 +29,34 @@ data class RawEntity(
     override fun unwrap(): Referencable {
         val entity = RawEntity(
             id = id,
+            creationTimestamp = creationTimestamp,
             expirationTimestamp = expirationTimestamp,
             singletons = singletons.mapValues { it.value?.unwrap() },
             collections = collections.mapValues {
                 it.value.map { item -> item.unwrap() }.toSet()
             }
         )
+        entity.creationTimestamp = creationTimestamp
         entity.expirationTimestamp = expirationTimestamp
         return entity
     }
 
+    /** Entity creation time (in millis). */
+    @Suppress("GoodTime") // use Instant
+    override var creationTimestamp: Long = UNINITIALIZED_TIMESTAMP
+        set(value) {
+            require(this.creationTimestamp == UNINITIALIZED_TIMESTAMP) {
+                "cannot override creationTimestamp $value"
+            }
+            @Suppress("GoodTime") // use Instant
+            field = value
+        }
+
     /** Entity expiration time (in millis). */
     @Suppress("GoodTime") // use Instant
-    override var expirationTimestamp: Long = NO_EXPIRATION
+    override var expirationTimestamp: Long = UNINITIALIZED_TIMESTAMP
         set(value) {
-            require(this.expirationTimestamp == NO_EXPIRATION) {
+            require(this.expirationTimestamp == UNINITIALIZED_TIMESTAMP) {
                 "cannot override expirationTimestamp $value"
             }
             @Suppress("GoodTime") // use Instant
@@ -62,7 +75,8 @@ data class RawEntity(
         id: ReferenceId = NO_REFERENCE_ID,
         singletonFields: Set<FieldName> = emptySet(),
         collectionFields: Set<FieldName> = emptySet(),
-        expirationTimestamp: Long = NO_EXPIRATION
+        creationTimestamp: Long = UNINITIALIZED_TIMESTAMP,
+        expirationTimestamp: Long = UNINITIALIZED_TIMESTAMP
     ) : this(
         id,
         singletonFields.associateWith { null },
@@ -73,17 +87,21 @@ data class RawEntity(
 
     companion object {
         const val NO_REFERENCE_ID = "NO REFERENCE ID"
-        const val NO_EXPIRATION: Long = -1
+        const val UNINITIALIZED_TIMESTAMP: Long = -1
     }
 }
 
 fun RawEntity(
     id: String,
     singletons: Map<FieldName, Referencable?>,
     collections: Map<FieldName, Set<Referencable>>,
+    creationTimestamp: Long,
     expirationTimestamp: Long
 ) = RawEntity(
     id,
     singletons,
     collections
-).also { it.expirationTimestamp = expirationTimestamp }
+).also {
+    it.creationTimestamp = creationTimestamp
+    it.expirationTimestamp = expirationTimestamp
+}

java/arcs/core/data/Ttl.kt

@@ -34,7 +34,7 @@ sealed class Ttl(count: Int, val isInfinite: Boolean = false) {
     override fun hashCode(): Int = minutes.hashCode()
 
     fun calculateExpiration(time: Time): Long =
-        if (isInfinite) RawEntity.NO_EXPIRATION
+        if (isInfinite) RawEntity.UNINITIALIZED_TIMESTAMP
         else requireNotNull(time).currentTimeMillis + (minutes * 60 * 1000)
 
     data class Minutes(val count: Int) : Ttl(count)

java/arcs/core/data/proto/TypeProtoDecoders.kt

@@ -11,10 +11,11 @@
 
 package arcs.core.data.proto
 
+import arcs.core.data.FieldType
 import arcs.core.data.PrimitiveType
 
 /**
- * Converts a [PrimitiveTypeProto] protobuf instance into a native kotlin [PrimitiveType] instance.
+ * Converts a [PrimitiveTypeProto] protobuf instance into a Kotlin [PrimitiveType] instance.
  */
 fun PrimitiveTypeProto.decode(): PrimitiveType =
     when (this) {
@@ -24,3 +25,25 @@ fun PrimitiveTypeProto.decode(): PrimitiveType =
         PrimitiveTypeProto.UNRECOGNIZED ->
             throw IllegalArgumentException("Unknown PrimitiveTypeProto value.")
     }
+
+/**
+ * Converts a [PrimitiveTypeProto] protobuf instance into a Kotlin [FieldType] instance.
+ */
+fun PrimitiveTypeProto.decodeAsFieldType(): FieldType.Primitive = FieldType.Primitive(decode())
+
+/**
+ * Converts a [TypeProto] protobuf instance into a Kotlin [FieldType] instance.
+ *
+ * @throws [IllegalArgumentexception] if the type cannot be converted to [FieldType].
+ */
+fun TypeProto.decodeAsFieldType(): FieldType =
+    when (getDataCase()) {
+        TypeProto.DataCase.PRIMITIVE -> getPrimitive().decodeAsFieldType()
+        // TODO: Handle FieldType.EntityRef. It is not clear how it is
+        // represented in the proto.
+        TypeProto.DataCase.DATA_NOT_SET ->
+            throw IllegalArgumentException("Unknown data field in TypeProto.")
+        else ->
+            throw IllegalArgumentException(
+                "Cannot decode a ${getDataCase().name} type to a [FieldType].")
+    }

java/arcs/core/storage/handle/CollectionImpl.kt

@@ -88,6 +88,8 @@ class CollectionImpl<T : Referencable>(
      */
     suspend fun store(entity: T): Boolean {
         log.debug { "Storing: $entity" }
+        @Suppress("GoodTime") // use Instant
+        entity.creationTimestamp = requireNotNull(time).currentTimeMillis
         if (!Ttl.Infinite.equals(ttl)) {
             @Suppress("GoodTime") // use Instant
             entity.expirationTimestamp = ttl.calculateExpiration(time)

java/arcs/core/storage/handle/SingletonImpl.kt

@@ -64,6 +64,8 @@ class SingletonImpl<T : Referencable>(
      * did not apply fully. Fetch the latest value and retry.
      * */
     suspend fun store(entity: T): Boolean {
+        @Suppress("GoodTime") // use Instant
+        entity.creationTimestamp = requireNotNull(time).currentTimeMillis
         if (!Ttl.Infinite.equals(ttl)) {
             @Suppress("GoodTime") // use Instant
             entity.expirationTimestamp = ttl.calculateExpiration(time)

java/arcs/sdk/android/dev/api/CollectionProxy.java

@@ -286,6 +286,15 @@ public Referencable unwrap() {
       return this;
     }
 
+    @Override
+    public long getCreationTimestamp() {
+      throw new AssertionError("ModelEntry::getCreationTimestamp not implemented");
+    }
+
+    @Override
+    public void setCreationTimestamp(long creationTimestamp) {
+      throw new AssertionError("ModelEntry::setCreationTimestamp not implemented");
+    }
 
     @Override
     public long getExpirationTimestamp() {

java/arcs/sdk/android/storage/ServiceStore.kt

@@ -142,12 +142,15 @@ class ServiceStore<Data : CrdtData, Op : CrdtOperation, ConsumerData>(
         val service = connection.connectAsync().await()
 
         val messageChannel = ParcelableProxyMessageChannel(coroutineContext)
-        serviceCallbackToken = withContext(coroutineContext) {
-            service.registerCallback(messageChannel)
-        }
 
+        // Open subscription before attaching callback to make sure that we capture all messages
+        val subscription = messageChannel.openSubscription()
         scope.launch {
-            messageChannel.openSubscription().consumeEach { handleMessageAndResultFromService(it) }
+            subscription.consumeEach { handleMessageAndResultFromService(it) }
+        }
+
+        serviceCallbackToken = withContext(coroutineContext) {
+            service.registerCallback(messageChannel)
         }
 
         this.serviceConnection = connection

javatests/arcs/android/host/AndroidAllocatorTest.kt

@@ -13,7 +13,6 @@ package arcs.android.host
 
 import android.content.Context
 import androidx.lifecycle.Lifecycle
-import androidx.test.core.app.ActivityScenario
 import androidx.test.core.app.ApplicationProvider
 import androidx.test.ext.junit.runners.AndroidJUnit4
 import androidx.work.testing.WorkManagerTestInitHelper
@@ -26,12 +25,9 @@ import kotlinx.coroutines.ExperimentalCoroutinesApi
 import kotlinx.coroutines.GlobalScope
 import kotlinx.coroutines.launch
 import kotlinx.coroutines.runBlocking
-import kotlinx.coroutines.test.TestCoroutineScope
-import kotlinx.coroutines.test.runBlockingTest
 import org.junit.Before
 import org.junit.runner.RunWith
 import org.robolectric.Robolectric
-import kotlin.coroutines.CoroutineContext
 
 /**
  * These tests are the same as [AllocatorTestBase] but run with Android Services,
@@ -82,24 +78,4 @@ open class AndroidAllocatorTest : AllocatorTestBase() {
         writingService = Robolectric.setupService(TestWritingExternalHostService::class.java)
         super.setUp()
     }
-
-    override fun runAllocatorTest(
-        coroutineContext: CoroutineContext,
-        testBody: suspend TestCoroutineScope.() -> Unit
-    ) = runBlockingTest(coroutineContext) {
-        val scenario = ActivityScenario.launch(TestActivity::class.java)
-
-        scenario.moveToState(Lifecycle.State.STARTED)
-
-        val activityJob = launch {
-            scenario.onActivity { activity ->
-                runBlocking {
-                    this@runBlockingTest.testBody()
-                }
-                scenario.close()
-            }
-        }
-
-        activityJob.join()
-    }
 }

javatests/arcs/android/host/AndroidEntityHandleManagerTest.kt

@@ -2,7 +2,8 @@ package arcs.android.host
 
 import android.app.Application
 import androidx.lifecycle.Lifecycle
-import androidx.test.core.app.ActivityScenario
+import androidx.lifecycle.LifecycleOwner
+import androidx.lifecycle.LifecycleRegistry
 import androidx.test.core.app.ApplicationProvider
 import androidx.test.ext.junit.runners.AndroidJUnit4
 import androidx.work.testing.WorkManagerTestInitHelper
@@ -27,8 +28,6 @@ import arcs.sdk.android.storage.service.testutil.TestBindingDelegate
 import com.google.common.truth.Truth.assertThat
 import kotlinx.coroutines.launch
 import kotlinx.coroutines.runBlocking
-import kotlinx.coroutines.test.TestCoroutineScope
-import kotlinx.coroutines.test.runBlockingTest
 import org.junit.Before
 import org.junit.Test
 import org.junit.runner.RunWith
@@ -38,12 +37,15 @@ typealias Person = TestParticleInternal1
 
 @Suppress("EXPERIMENTAL_API_USAGE")
 @RunWith(AndroidJUnit4::class)
-class AndroidEntityHandleManagerTest {
+class AndroidEntityHandleManagerTest : LifecycleOwner {
     private lateinit var app: Application
+    private lateinit var lifecycle: LifecycleRegistry
+    override fun getLifecycle() = lifecycle
 
     val entity1 = Person("Jason", 21.0, false)
     val entity2 = Person("Jason", 22.0, true)
     lateinit var handleHolder: TestParticleHandles
+    private lateinit var handleManager: EntityHandleManager
 
     private val schema = Schema(
         listOf(SchemaName("Person")),
@@ -69,37 +71,24 @@ class AndroidEntityHandleManagerTest {
     @Before
     fun setUp() {
         app = ApplicationProvider.getApplicationContext()
-        app.setTheme(R.style.Theme_AppCompat);
+        lifecycle = LifecycleRegistry(this).apply {
+            setCurrentState(Lifecycle.State.CREATED)
+            setCurrentState(Lifecycle.State.STARTED)
+            setCurrentState(Lifecycle.State.RESUMED)
+        }
 
         // Initialize WorkManager for instrumentation tests.
         WorkManagerTestInitHelper.initializeTestWorkManager(app)
 
         handleHolder = TestParticleHandles()
-    }
 
-    fun handleManagerTest(
-        block: suspend TestCoroutineScope.(EntityHandleManager) -> Unit
-    ) = runBlockingTest {
-        val scenario = ActivityScenario.launch(TestActivity::class.java)
-
-        scenario.moveToState(Lifecycle.State.STARTED)
-
-        val activityJob = launch {
-            scenario.onActivity { activity ->
-                val hf = AndroidHandleManager(
-                    lifecycle = activity.lifecycle, context = activity,
-                    connectionFactory = DefaultConnectionFactory(
-                        activity, TestBindingDelegate(app), coroutineContext
-                    ),
-                    coroutineContext = coroutineContext
-                )
-                runBlocking {
-                    this@runBlockingTest.block(EntityHandleManager(hf))
-                }
-                scenario.close()
-            }
-        }
-        activityJob.join()
+        handleManager = EntityHandleManager(
+            AndroidHandleManager(
+                lifecycle = lifecycle,
+                context = app,
+                connectionFactory = DefaultConnectionFactory(app, TestBindingDelegate(app))
+            )
+        )
     }
 
     private fun expectHandleException(handleName: String, block: () -> Unit) {
@@ -110,7 +99,7 @@ class AndroidEntityHandleManagerTest {
     }
 
     @Test
-    fun handle_uninitializedThrowsException() = handleManagerTest {
+    fun handle_uninitializedThrowsException() = runBlocking {
         expectHandleException("writeHandle") {
             handleHolder.writeHandle
         }
@@ -161,7 +150,7 @@ class AndroidEntityHandleManagerTest {
     )
 
     @Test
-    fun singletonHandle_writeFollowedByReadWithOnUpdate() = handleManagerTest { handleManager ->
+    fun singletonHandle_writeFollowedByReadWithOnUpdate() = runBlocking {
         val writeHandle = createSingletonHandle(
             handleManager,
             "writeHandle",
@@ -208,7 +197,7 @@ class AndroidEntityHandleManagerTest {
     }
 
     @Test
-    fun setHandle_writeFollowedByReadWithOnUpdate() = handleManagerTest { handleManager ->
+    fun setHandle_writeFollowedByReadWithOnUpdate() = runBlocking<Unit> {
         val writeSetHandle = createSetHandle(
             handleManager,
             "writeSetHandle",

javatests/arcs/android/host/AndroidManifest.xml

@@ -15,12 +15,6 @@
       </intent-filter>
     </service>
     <service android:name="arcs.sdk.android.service.StorageService" android:exported="false"/>
-    <activity android:name="arcs.android.host.TestActivity" >
-      <intent-filter>
-        <action android:name="android.intent.action.MAIN"/>
-        <category android:name="android.intent.category.LAUNCHER"/>
-      </intent-filter>
-    </activity>
   </application>
 
 </manifest>