Skip to content

Including handle in the submission #46

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
lstkz opened this issue Jan 18, 2017 · 6 comments
Open

Including handle in the submission #46

lstkz opened this issue Jan 18, 2017 · 6 comments

Comments

@lstkz
Copy link

lstkz commented Jan 18, 2017

The review process is anonymous by default, and members shouldn't reveal
their identity anywhere.
However, members can be recognized in an easy way by:

  • git patch, youtube username, monosnap username
  • using TC API
  • submissions are not hidden on the challenge page
@birdofpreyru
Copy link

It makes no sense. Because if a competitor and a reviewer want to hack the rules, the competitor can just privately email the ID of his submission to the reviewer. Thus, you may close any other ways for a reviewer to figure out the identity of competitors, but you'll never be able to prevent this simple way to hack the system.

The only way to deal with this is: if a review is not fair, other competitors should bring it up to the co-pilots/managers/community, the issue should be considered in public, and all involved in cheating and/or unfair judgement should be strictly punished.

@ThomasKranitsas
Copy link

Totally agree with @birdofpreyru !

@lstkz
Copy link
Author

lstkz commented Jan 18, 2017

It's not related to cheating because we can't prevent it.

See last post https://apps.topcoder.com/forums/?module=Thread&threadID=623369&start=0&mc=8#1021109

But for instance if I knew I was reviewing a design by Pops I would be expecting it to score very highly; if my review score was less than 95 I'd be worried I had made a mistake. And if I knew the design was by a guy with a rating of 700, I would be expecting it to be poor.

@birdofpreyru
Copy link

Anyway, I don't like this because:

  1. Demands extra efforts from a competitor to hide his identity (e.g. I use Linux, and in all my demo/verification videos you can see my Terminal Console with my username in the path, do you want me to hide it somehow? I have no idea, how to do it, it sure demands extra efforts). Similar thing applies to usernames in various services, etc.

using TC API
submissions are not hidden on the challenge page

These demands some changes to the codebase, i.e. $$ to be paid and work to be done.

Thus, needs efforts from all parts, while the impact is not obvious for me, I believe in 95% of cases, any initial bias does not change the outcome of reviews, while the rest 5% are probably closer to cheating, and won't be prevented by these efforts.

@lstkz
Copy link
Author

lstkz commented Jan 19, 2017

@topcoderinc topcoderinc locked and limited conversation to collaborators Jan 19, 2017
@hokienick
Copy link
Contributor

We need to discuss a way for us to create a process/rules/policy to keep anonymity

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

4 participants