-
Notifications
You must be signed in to change notification settings - Fork 13
Add legal hold membership to device reporting #192
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 6 commits
b285a68
ed72879
ea4381a
a0ebe57
077dea1
03a3814
f174992
205d559
02d6530
e5784cf
9bfcd42
2e303de
94fe3b9
0b79911
1bd1e2f
e4725c7
237ea31
7212fbf
2a917bd
2d1db8c
9a0afcb
a3dd28f
9677f23
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -5,6 +5,7 @@ | |
|
||
import click | ||
from click import echo | ||
from pandas import DataFrame | ||
|
||
from code42cli.bulk import generate_template_cmd_factory | ||
from code42cli.bulk import run_bulk_process | ||
|
@@ -92,8 +93,19 @@ def _list(state, format=None): | |
is_flag=True, | ||
help="View details of the preservation policy associated with the legal hold matter.", | ||
) | ||
@click.option( | ||
"--include-devices", | ||
is_flag=True, | ||
help="View devices and storage associated with legal hold custodians.", | ||
) | ||
@sdk_options() | ||
def show(state, matter_id, include_inactive=False, include_policy=False): | ||
def show( | ||
state, | ||
matter_id, | ||
include_inactive=False, | ||
include_policy=False, | ||
include_devices=False, | ||
): | ||
"""Display details of a given legal hold matter.""" | ||
matter = _check_matter_is_accessible(state.sdk, matter_id) | ||
matter["creator_username"] = matter["creator"]["username"] | ||
|
@@ -105,19 +117,32 @@ def show(state, matter_id, include_inactive=False, include_policy=False): | |
memberships = _get_legal_hold_memberships_for_matter( | ||
state.sdk, matter_id, active=active | ||
) | ||
active_usernames = [ | ||
member["user"]["username"] for member in memberships if member["active"] | ||
] | ||
inactive_usernames = [ | ||
member["user"]["username"] for member in memberships if not member["active"] | ||
] | ||
|
||
formatter = OutputFormatter(OutputFormat.TABLE, _MATTER_KEYS_MAP) | ||
formatter.echo_formatted_list([matter]) | ||
_print_matter_members(active_usernames, member_type="active") | ||
|
||
users = [ | ||
[member["active"], member["user"]["userUid"], member["user"]["username"]] | ||
for member in memberships | ||
] | ||
|
||
usernames = [user[2] for user in users if user[0] is True] | ||
_print_matter_members(usernames, member_type="active") | ||
if include_inactive: | ||
_print_matter_members(inactive_usernames, member_type="inactive") | ||
usernames = [user[2] for user in users if user[0] is not True] | ||
_print_matter_members(usernames, member_type="inactive") | ||
|
||
if include_devices: | ||
user_dataframe = _build_user_dataframe(users) | ||
devices_dataframe = _merge_matter_members_with_devices( | ||
state.sdk, user_dataframe | ||
) | ||
if len(devices_dataframe.index) > 0: | ||
maddie-vargo marked this conversation as resolved.
Show resolved
Hide resolved
|
||
echo("\nMatter Members and Devices:\n") | ||
click.echo(devices_dataframe.to_csv()) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We typically use table format for show commands. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. List commands, however, are known to support various output formats, inc. CSV.... My first comment on this PR suggests that this would make sense as a |
||
echo(_print_storage_by_org(devices_dataframe)) | ||
else: | ||
echo("\nNo devices associated with matter.\n") | ||
|
||
if include_policy: | ||
_get_and_print_preservation_policy(state.sdk, matter["holdPolicyUid"]) | ||
|
@@ -238,6 +263,50 @@ def _print_matter_members(username_list, member_type="active"): | |
echo("No {} matter members.\n".format(member_type)) | ||
|
||
|
||
def _merge_matter_members_with_devices(sdk, user_dataframe): | ||
devices_generator = sdk.devices.get_all(active="true", include_backup_usage=True) | ||
maddie-vargo marked this conversation as resolved.
Show resolved
Hide resolved
|
||
device_list = _get_total_archive_bytes_per_device(devices_generator) | ||
devices_dataframe = DataFrame.from_records( | ||
device_list, | ||
columns=[ | ||
"userUid", | ||
"guid", | ||
"name", | ||
"osHostname", | ||
"status", | ||
"alertStates", | ||
"orgId", | ||
"lastConnected", | ||
"version", | ||
"archiveBytes", | ||
], | ||
) | ||
return user_dataframe.merge( | ||
devices_dataframe, how="inner", on="userUid" | ||
).reset_index(drop=True) | ||
|
||
|
||
def _build_user_dataframe(users): | ||
user_dataframe = DataFrame.from_records( | ||
users, columns=["activeMembership", "userUid", "username"] | ||
) | ||
return user_dataframe | ||
|
||
|
||
def _get_total_archive_bytes_per_device(devices_generator): | ||
device_list = [device for page in devices_generator for device in page["computers"]] | ||
for i in device_list: | ||
maddie-vargo marked this conversation as resolved.
Show resolved
Hide resolved
|
||
archive_bytes = [archive["archiveBytes"] for archive in i["backupUsage"]] | ||
i["archiveBytes"] = sum(archive_bytes) | ||
return device_list | ||
|
||
|
||
def _print_storage_by_org(devices_dataframe): | ||
maddie-vargo marked this conversation as resolved.
Show resolved
Hide resolved
|
||
echo("\nLegal Hold Storage by Org\n") | ||
devices_dataframe = devices_dataframe.filter(["orgId", "archiveBytes"]) | ||
return devices_dataframe.groupby("orgId").sum() | ||
|
||
|
||
@lru_cache(maxsize=None) | ||
def _check_matter_is_accessible(sdk, matter_id): | ||
return sdk.legalhold.get_matter_by_uid(matter_id) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We do have a new
devices list
command that is merged by not yet released.Would it make more sense to have a command
code42 devices list --matter-id <UID>
instead?Just want to make sure all the options are considered, I don't really understand the use-case.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's an interesting point. I chose to add it to
legal-hold show
because that command already showed details on a legal hold matter object, including users on legal hold and (optional) preservation policy details. I thought showing devices on legal hold would be a logical fit here.From what I gather, the
devices
command renders mostly theComputer
api resource. Unfortunately, there is no flag in theComputer
api response that states whether a device is on legal hold. In order to pull legal hold devices, you first need to pull legal hold membership to get a list of users and then find the devices associated with those users. It would not be impossible to move this into thedevices
command, but we would still have to run some sort of legal hold py42 resource to determine the scope of devices.Pinging @ceciliastevens for input as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Or a
code42 legal-hold list-devices [MATTER-ID]
. Trying to find a way to afford output format options (At least table and csv).There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree, if we can find a way to structure this to add a format option, that would be helpful to most customers.
Currently, the
legal-hold
show command outputs data in the following formats:What do you think about removing users from the
show
command and creatingcode42 legal-hold list-users [Matter ID]
with an--include-devices
optiion? I think it makes sense to keep users and devices closely tied together, rather than in separate commands. From a code perspective, to get devices, you'll have to get users first, anyway.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like that! @timabrmsn What are you thoughts on this? I believe you were the one who implemented the show command.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I actually think it makes more sense to add legal hold status as an option to the
devices list
command, since that would be a useful option for people who are creating a report of all their devices to know which ones are on legal hold quickly when taking inventory of other things. It could just add alegalHoldMatterId
column to the output.And keeping users under
legal-hold show
still makes sense to me, since it's a high level overview of the matter, of which membership is a key detail. Otherwise theshow
command just becomes a single row of thelist
command with the option to also show preservation policy.devices list
already has--include-usernames
which makes a separate call toapi/Users
and then merges the DataFrames onuserId
, so adding legal hold status would be easy enough to add using the same logic.The storage total report is an interesting idea, but might be better suited to include in
devices list
report also, so it can be used more generically and not tied directly to only legal hold. I'm thinking something like--include-storage-totals
ondevices list
that just adds a column per device for the sum of all that device's archives. Then if someone wanted to get a report broken down by matter Id, they could just filter by matter_id and sum the total in the csv report.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok,
devices
it is. The device pull actually includesuserUid
without the username option, so it's even easier to merge in the legal hold data.There are two main things that we want this to accomplish:
Limit the device list to devices on a specific legal hold matter
devices --matter-uid
- option to limit device search base on a specific legal hold matter, adds in columns such as legalHoldStatus, legalHoldMatterId, legalHoldMatterNameAdd in legal hold membership columns for whatever device query you're running.
devices -include-legal-hold-membership
- option to option to include legal hold membership on to the devices table, adds same columns as above but doesn't limit the populationThis feels a little redundant, so any feedback is appreciated.
As for
storage-totals
: is a sum of all device's archives appropriate? (this is how I wrote my first draft). However, the current legal hold script does some logic to determine which destination is the most recent and complete, and only uses that archive size as the total storage.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think that since filtering by matter-id outside of the CLI is simple enough (i.e. grep/Select-String or directly in excel), can we leave that to the end-user? I'd like to not have our option list expand too much. And I have some ideas for more generic column/row filtering logic that could work on any cmd that outputs a dataframe. Something like `--filter 'matter-id == 42' that would convert to a pandas expression under the hood. Although I haven't started trying to implement that so I'm not sure how feasible it is...
Also, because a device can be part of multiple legal holds, we need to think of how to display that. Can we just have a single column that has a list of matter-uids the device is part of, maybe separated by semi-colons or something?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, for storage totals I think adding a
usedStorage
column that sums the total combined archive size for a device would work, maybe we could also add anarchiveCount
column that gets added with that option?Trying to keep what features we add as generic and multi-use-case friendly as possible.