[$250] Admin plugin? #14

jmgasper · 2020-05-05T23:41:35Z

@atelomycterus - Topcoder needs to comply with some GDPR and security requirements. To make that easy, we've been asked to investigate if we can create an admin plugin that allows an admin to:

Admin have the ability to identify all messages from a user
Platform must support the ability to export data on a per-user basis
Must log who posted a message, date, time, and contents if text (seems like this is already done by default - I'm getting clarification)

Idea

What I'm thinking (and I'm not sure if this is possible) is that we create a new plugin (or extend the existing one) to add a new tab and / or new options to an existing tab, on the dashboard site with the required functionality:

jmgasper · 2020-05-05T23:43:24Z

Contest https://www.topcoder.com/challenges/30124447 has been created for this ticket.

This is an automated message for ghostar via Topcoder X

jmgasper · 2020-05-05T23:43:45Z

Contest https://www.topcoder.com/challenges/30124447 has been updated - it has been assigned to obog.

This is an automated message for ghostar via Topcoder X

atelomycterus · 2020-05-06T07:21:39Z

@jmgasper It is necessary to clarify some details in order to understand how much work is required,
whether the required data is saved in the forum, or it is required to implement new functionality.

Vanilla provides default logging for some functionality. I am worried that, for example, if we need to save data for 3 months or more, then maybe with large amounts of data this can affect system performance. Perhaps we need to make your own history table, it depends on requirements. Vanilla uses caching in some cases, but actively uses tables that store messages and notifications. The default settings are to delete data and truncate tables after a few weeks. What period should we store data?
Admin have the ability to identify all messages from a user

What did you mean by all messages? Did you mean any user generated conent?
if yes, then user generated content in Vanilla:

chats between users (messages)
forums (discussions/threads/comments)

I think that there are no problems and the data can be easily identified by user.

Platform must support the ability to export data on a per-user basis

I haven't seen any export data functionality in Vanilla. I'll check it.

I think the following data should be exported on a per-user basis:

discussions/threads/comments
chat messages. If we plan to use internal chats
user profile (last name, first name, email)
IP/datetime when user logged in? It is necessary to check, but perhaps only the last login date and IP are stored.

Please correct me if I am wrong or some data missing.

I will provide information in which tables and what format the data is stored.
Some data is stored as templates (e.g. '{userId} wrote {message}') and template parsing is performed during page loading/rendering.
Perhaps here some difficulties will arise, because Vanilla provides the ability to modify content and modify data in plugins using hooks.

Must log who posted a message, date, time, and contents if text (seems like this is already done by default - I'm getting clarification)

Vanilla logs some data. As far as I remember, there is no data logging in some cases,
for example, the author has changed the message. The message table contains the creation date, update date, author, message body
but the previous message body is not logged. The previous body is logged if the message is edited by another user who has the role of a moderator.

What about the "Right to be forgotten" GDPR rule?
It means to give users the ability to get their account data removed on request
Do you plan to remove data manually? This task can be automated:

an user data (email and so on) should be replaced with a new specific one because there might be discussions/posts/comments/messages and so on
replace a profile image with a default Vanilla icon
username/password need to be changed because user could try to log in
all notifications should be set to be off
maybe something else.

Implementation
I'll look through the source code to understand how best to implement it as a plugin or as an application.
Keep you posted.

jmgasper · 2020-05-06T07:42:41Z

The default is to remove public messages from the forum? We'll have to disable that completely, if so.
This would be chats and forum content - basically anything generated by a given user.
That should work for now. I don't think we need IP / login date information.
We can just export raw data, as long as it captures what the user has input. I don't really think it matters if it's very "user friendly", as long as the data is there.
I'll ask about "forgotten" rights and what that entails.

atelomycterus · 2020-05-06T17:06:53Z

@jmgasper

The default is to remove public messages from the forum? We'll have to disable that completely, if so.

It became clear how it works. Vanilla saves discussions into the 'discussion' table, comments into the 'comment' table and so on. Vanilla uses an auxiliary table called 'Activity' that stores activity data. This table is pruned periodically. It might be configured with 'Garden.PruneActivityAfter' (2 monthes
by default). Actual data is not deleted from tables (discussion, comments and so on) when the 'Activity' table is pruned.

For example, User creates a comment. On the backend side:

the comment will be added in the comment table. The table contains the list of columns the insertUserID, updateUserID, insertDate, updateDate, body and so on.
the template (e.g. '{userid} added the comment') will be added into the activity table. This data will be parsed and displayed in notification popups.

Most of the required data can be exported using the REST API.
We can call the appropriate REST methods, convert the data to a suitable format and generate a report.

Discussions
Get all discussions created by User:
http://127.0.0.1/settings/swagger#/Discussions/get_discussions
http://127.0.0.1/api/v2/discussions
Query parameters
insertUserID - Filter by author.
dateInserted - Filter by insert date
dateUpdated - Filter by update date

We can filter based on a range of date values. Read about date filters - https://success.vanillaforums.com/kb/articles/45-date-filters.

Output of the REST API method:

[
  {
    "discussionID": 6,
    "type": null,
    "name": "Tech requirements",
    "body": "<p>The list of all tech requirements:</p><ul><li>Requirement 1</li><li>Requirement 2</li><li>Requirement 3</li><li>Requirement 4</li></ul>",
    "categoryID": 17,
    "dateInserted": "2020-05-06T13:43:05+00:00",
    "dateUpdated": null,
    "insertUserID": 2,
    "pinned": false,
    "pinLocation": null,
    "closed": false,
    "sink": false,
    "countComments": 0,
    "countViews": 1,
    "score": null,
    "url": "http://127.0.0.1/discussion/6/tech-requirements",
    "canonicalUrl": "http://127.0.0.1/discussion/6/tech-requirements",
    "bookmarked": false,
    "unread": false
  }
]

Drafts (not published discussion)
Rest API /drafts - get drafts only for CURRENT user. Need to expand REST API to filter by userID, dates.

[
  {
    "draftID": 2,
    "recordType": "discussion",
    "parentRecordID": 17,
    "attributes": {
      "name": "Tech requirements",
      "tags": null,
      "closed": 0,
      "announce": 0,
      "sink": 0,
      "body": "[{\"insert\":\"The list of all tech requirements:\\nRequirement 1\"},{\"attributes\":{\"list\":{\"depth\":0,\"type\":\"bullet\"}},\"insert\":\"\\n\"},{\"insert\":\"Requirement 2\"},{\"attributes\":{\"list\":{\"depth\":0,\"type\":\"bullet\"}},\"insert\":\"\\n\"},{\"insert\":\"Requirement 3\"},{\"attributes\":{\"list\":{\"depth\":0,\"type\":\"bullet\"}},\"insert\":\"\\n\"},{\"insert\":\"Requirement 4\"},{\"attributes\":{\"list\":{\"depth\":0,\"type\":\"bullet\"}},\"insert\":\"\\n\"}]",
      "format": "Rich"
    },
    "insertUserID": 2,
    "dateInserted": "2020-05-06T13:43:17+00:00",
    "updateUserID": 2,
    "dateUpdated": "2020-05-06T13:43:17+00:00"
  }
]

Comments
http://127.0.0.1/settings/swagger#/Comments/get_comments
http://127.0.0.1/api/v2/comments
Query parameters
insertUserID - Filter by author.
dateInserted Filter by insert date
dateUpdated Filter by update date

Output:

[
  {
    "commentID": 1,
    "discussionID": 1,
    "body": "<div class=\"js-embed embedResponsive\" data-embedJson=\"{&quot;recordID&quot;:1,&quot;recordType&quot;:&quot;discussion&quot;,&quot;body&quot;:&quot;&lt;p&gt;public body&lt;\\/p&gt;&quot;,&quot;bodyRaw&quot;:[{&quot;insert&quot;:&quot;public body\\n&quot;}],&quot;format&quot;:&quot;Rich&quot;,&quot;dateInserted&quot;:&quot;2020-05-06T10:23:03+00:00&quot;,&quot;insertUser&quot;:{&quot;userID&quot;:2,&quot;name&quot;:&quot;obog&quot;,&quot;photoUrl&quot;:&quot;http:\\/\\/127.0.0.1\\/applications\\/dashboard\\/design\\/images\\/defaulticon.png&quot;,&quot;dateLastActive&quot;:&quot;2020-05-01T16:41:23+00:00&quot;},&quot;displayOptions&quot;:{&quot;showUserLabel&quot;:false,&quot;showCompactUserInfo&quot;:true,&quot;showDiscussionLink&quot;:true,&quot;showPostLink&quot;:true,&quot;showCategoryLink&quot;:false,&quot;renderFullContent&quot;:false,&quot;expandByDefault&quot;:false},&quot;url&quot;:&quot;http:\\/\\/127.0.0.1\\/discussion\\/1\\/public-thread&quot;,&quot;embedType&quot;:&quot;quote&quot;,&quot;name&quot;:&quot;Public  thread&quot;}\">\n    <a href=\"http://127.0.0.1/discussion/1/public-thread\">\n        http://127.0.0.1/discussion/1/public-thread\n    </a>\n</div><p>Quoted</p>",
    "dateInserted": "2020-05-06T10:23:41+00:00",
    "dateUpdated": null,
    "insertUserID": 2,
    "score": null,
    "url": "http://127.0.0.1/discussion/comment/1#Comment_1"
  },
  {
    "commentID": 2,
    "discussionID": 3,
    "body": "<p>I have some questions.</p>",
    "dateInserted": "2020-05-06T13:37:08+00:00",
    "dateUpdated": null,
    "insertUserID": 2,
    "score": null,
    "url": "http://127.0.0.1/discussion/comment/2#Comment_2"
  }
]

Conversations and Messages
Get all conversations with messages:
http://127.0.0.1/settings/swagger#/Conversations/get_conversations
http://127.0.0.1/api/v2/conversations
Query parameters
insertUserID - Filter by author.
participantUserID Filter by participating user.
dateInserted Filter by insert date
dateUpdated Filter by update date

Example of conversations:

[
  {
    "conversationID": 1,
    "name": "tonyj",
    "body": "<p>Hello again</p>",
    "url": "http://127.0.0.1/messages/1",
    "dateInserted": "2020-05-06T14:27:53+00:00",
    "insertUserID": 3,
    "countParticipants": 2,
    "participants": [
      {
        "userID": 3,
        "user": {
          "userID": 3,
          "name": "tonyj",
          "photoUrl": "https://topcoder-dev-media.s3.amazonaws.com/member/profile/TonyJ-1558928928786.png",
          "dateLastActive": "2020-05-06T13:44:56+00:00"
        },
        "status": "participating"
      },
      {
        "userID": 2,
        "user": {
          "userID": 2,
          "name": "obog",
          "photoUrl": "http://127.0.0.1/applications/dashboard/design/images/defaulticon.png",
          "dateLastActive": "2020-05-06T14:42:09+00:00"
        },
        "status": "participating"
      }
    ],
    "countMessages": 3,
    "unread": false,
    "countUnread": 0
  }
]

Get messages by Author only
http://127.0.0.1/settings/swagger#/Messages/get_messages
http://127.0.0.1/api/v2/messages
Query parameters
insertUserID - Filter by author

Output:

[
  {
    "messageID": 3,
    "conversationID": 1,
    "body": "<p>Hello all</p>",
    "insertUserID": 2,
    "dateInserted": "2020-05-06T14:44:25+00:00"
  }
]

Users
http://127.0.0.1/settings/swagger#/Users/get_users__id_
http://127.0.0.1/api/v2/users/{userid}

{
  "userID": 2,
  "name": "obog",
  "email": "[email protected]",
  "photoUrl": "http://127.0.0.1/applications/dashboard/design/images/defaulticon.png",
  "points": 0,
  "emailConfirmed": true,
  "showEmail": false,
  "bypassSpam": false,
  "banned": 0,
  "dateInserted": "2020-05-01T16:41:23+00:00",
  "dateLastActive": "2020-05-06T14:42:09+00:00",
  "dateUpdated": "2020-05-06T14:17:34+00:00",
  "roles": [
    {
      "roleID": 16,
      "name": "Administrator"
    },
    {
      "roleID": 32,
      "name": "Moderator"
    }
  ]
}

No REST API for entities

a. Tagging allows users to add a tag to discussions they start in order to make them more discoverable. Admin settings has the 'Enable Tagging' option. It is disabled by default.

No REST API to get Tags by User. The tags is saved into the 'Tag' table.

b. System messages. The 'Message' table doesn't have any user-related columns

It is displayed:

c. Attachments
Attachments can be made to the following content:
-- discussions
-- comments

the 'Attachment' has the InsertUserID/UpdateUserID column.

d. the 'UserIP' table contains UserID, IPAddress, DateInserted, DateUpdated.
The GDPR states that IP addresses should be considered personal data as it enters the scope of ‘online identifiers’ ( See https://eugdprcompliant.com/personal-data/)

e. Media (images)
No REST API to search by media items. The 'Media' table has the 'insertUserID' column.

Implementation
It can be implemented as a plugin. Admin Dashboard fires navigation events ('init' and 'GetAppSettingsMenuItems'). So we can add a new tab with with permissions.

The tab configurations looks like

	"TabName" => [
            "permission" => [
			  "permission name"
            ],
            "section" => "TabName",
            "title" => "Tab name",
            "description" => "Tab descripton",
            "url" => "url to handle it",
            "empty" => true // no side menu
        ],

jmgasper · 2020-05-08T00:14:42Z

Ok, cool. We're more interested in the discussion content itself and not so much the activity

It sounds like we're in a good spot to get this implemented into the plugin. Do we need to clarify anything else?

jmgasper · 2020-05-08T00:16:10Z

Payment task has been updated: https://software.topcoder.com/review/actions/ViewProjectDetails?pid=30124447

This is an automated message for ghostar via Topcoder X

atelomycterus · 2020-05-08T06:04:35Z

@jmgasper We need to clarify only the 'forgotten' rule

I'll ask about "forgotten" rights and what that entails.

jmgasper added the tcx_OpenForPickup label May 5, 2020

jmgasper assigned atelomycterus May 5, 2020

jmgasper added tcx_Assigned and removed tcx_OpenForPickup labels May 5, 2020

atelomycterus added the tcx_ReadyForReview label May 6, 2020

jmgasper added tcx_FixAccepted and removed tcx_ReadyForReview labels May 8, 2020

jmgasper closed this as completed May 8, 2020

jmgasper added the tcx_Paid label May 8, 2020

jmgasper mentioned this issue May 8, 2020

[$450] Implement admin piece in TC plugin #15

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[$250] Admin plugin? #14

[$250] Admin plugin? #14

jmgasper commented May 5, 2020

jmgasper commented May 5, 2020

Uh oh!

jmgasper commented May 5, 2020

Uh oh!

atelomycterus commented May 6, 2020

Uh oh!

jmgasper commented May 6, 2020

Uh oh!

atelomycterus commented May 6, 2020 •

edited

Loading

Uh oh!

jmgasper commented May 8, 2020

Uh oh!

jmgasper commented May 8, 2020

Uh oh!

atelomycterus commented May 8, 2020 •

edited

Loading

Uh oh!

[$250] Admin plugin? #14

[$250] Admin plugin? #14

Comments

jmgasper commented May 5, 2020

Idea

jmgasper commented May 5, 2020

Uh oh!

jmgasper commented May 5, 2020

Uh oh!

atelomycterus commented May 6, 2020

Uh oh!

jmgasper commented May 6, 2020

Uh oh!

atelomycterus commented May 6, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jmgasper commented May 8, 2020

Uh oh!

jmgasper commented May 8, 2020

Uh oh!

atelomycterus commented May 8, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

atelomycterus commented May 6, 2020 •

edited

Loading

atelomycterus commented May 8, 2020 •

edited

Loading