Skip to content

Added A General Swish Activation Function inNeural Networks #10415

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
Oct 18, 2023
Merged

Added A General Swish Activation Function inNeural Networks #10415

merged 4 commits into from
Oct 18, 2023

Conversation

shivansh-bhatnagar18
Copy link
Contributor

Describe your change:

  • Add an algorithm?
  • Fix a bug or typo in an existing algorithm?
  • Documentation change?

Checklist:

  • I have read CONTRIBUTING.md.
  • This pull request is all my own work -- I have not plagiarized.
  • I know that pull requests will not be merged if they fail the automated tests.
  • This PR only changes one algorithm file. To ease review, please open separate PRs for separate algorithms.
  • All new Python files are placed inside an existing directory.
  • All filenames are in all lowercase characters with no spaces or dashes.
  • All functions and variable names follow Python naming conventions.
  • All function parameters and return values are annotated with Python type hints.
  • All functions have doctests that pass the automated testing.
  • All new algorithms include at least one URL that points to Wikipedia or another similar explanation.
  • If this pull request resolves one or more open issues then the description above includes the issue number(s) with a closing keyword: "Fixes #ISSUE-NUMBER".

@shivansh-bhatnagar18 shivansh-bhatnagar18 marked this pull request as ready for review October 14, 2023 06:30
@algorithms-keeper algorithms-keeper bot added the awaiting reviews This PR is ready to be reviewed label Oct 14, 2023
tianyizheng02
tianyizheng02 reviewed Oct 14, 2023
View reviewed changes
Copy link
Contributor

@tianyizheng02 tianyizheng02 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your contribution, but could you modify neural_network/activation_functions/sigmoid_linear_unit.py rather than making a separate file for the general swish function? Since the SiLU function is just the swish function with a parameter of 1, I think it'd make much more sense if you could rename that file to swish.py and generalize the existing function.

@shivansh-bhatnagar18
Copy link
Contributor Author

@tianyizheng02 Sir, I added the general swish model in the Sigmoid Linear Unit Model and renamed it as 'swish.py'. Now in order to maintain learning through these algorithms, I didn't find it appealing enough to delete the existing SiLU function to let others know that it is a popular name for the swish function of parameter 1. Hence I just added both functions. Could you please look into this and merge my PR?

@tianyizheng02 tianyizheng02 merged commit 572de4f into TheAlgorithms:master Oct 18, 2023
@tianyizheng02
Copy link
Contributor

Thanks for your contribution!

@algorithms-keeper algorithms-keeper bot removed the awaiting reviews This PR is ready to be reviewed label Oct 18, 2023
sedatguzelsemme pushed a commit to sedatguzelsemme/Python that referenced this pull request Sep 15, 2024
@shivansh-bhatnagar18 @shivanshbhatnagar18
@pre-commit-ci @sedatguzelsemme
Added A General Swish Activation Function inNeural Networks (TheAlgor…
250b566 
…ithms#10415)

* Added A General Swish Activation Function inNeural Networks

* Added the general swish function in the SiLU function and renamed it as swish.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: Shivansh Bhatnagar <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
@isidroas isidroas mentioned this pull request Jan 25, 2025
Closed
14 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tianyizheng02 tianyizheng02 tianyizheng02 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@shivansh-bhatnagar18 @tianyizheng02 @shivanshbhatnagar18