BUG: read_html() extracts only a single link from cells even if there are multiple in the same cell.

[kyoungrok0517]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd

# extract table
dfs = pd.read_html("https://en.wikipedia.org/wiki/Microsoft", extract_links="all") # dfs[0] = infobox

# Check the links of "Key people" row (11th)
# There are multiple links in the second column, but only the first one is extracted
print(dfs[0].iloc[10].values[1])

# ('Satya Nadella (Chairman and CEO)Brad Smith (vice chairman and president)Bill Gates (technical adviser)',
 '/en/Satya_Nadella')
# Only Satya Nadella's link is extracted, while other people are also hyperlinked.

Issue Description

read_html(extract_links="all") extracts only the first link from cells even if there are multiple in the same cell. For instance, if I parse Microsoft's Wikipedia page, even if there are several links contained in the same cell, only the first one is extracted, like below.

('Satya Nadella (Chairman and CEO)Brad Smith (vice chairman and president)Bill Gates (technical adviser)', '/en/Satya_Nadella')

Expected Behavior

Extract every links contained in the same cell.

Installed Versions

INSTALLED VERSIONS

commit : 0f43794
python : 3.10.6.final.0
python-bits : 64
OS : Darwin
OS-release : 22.6.0
Version : Darwin Kernel Version 22.6.0: Wed Jul 5 22:22:05 PDT 2023; root:xnu-8796.141.3~6/RELEASE_ARM64_T6000
machine : arm64
processor : arm
byteorder : little
LC_ALL : None
LANG : ko_KR.UTF-8
LOCALE : ko_KR.UTF-8

pandas : 2.0.3
numpy : 1.24.4
pytz : 2023.3
dateutil : 2.8.2
setuptools : 68.0.0
pip : 22.3.1
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : 3.1.2
lxml.etree : 4.9.2
html5lib : 1.1
pymysql : None
psycopg2 : None
jinja2 : 3.1.2
IPython : 8.7.0
pandas_datareader: None
bs4 : 4.12.2
bottleneck : None
brotli : 1.0.9
fastparquet : None
fsspec : 2023.6.0
gcsfs : None
matplotlib : 3.7.1
numba : 0.57.1
numexpr : 2.8.4
odfpy : None
openpyxl : 3.1.2
pandas_gbq : None
pyarrow : 12.0.1
pyreadstat : None
pyxlsb : None
s3fs : None
scipy : 1.11.1
snappy : None
sqlalchemy : 1.4.47
tables : None
tabulate : 0.9.0
xarray : None
xlrd : None
zstandard : 0.19.0
tzdata : 2023.3
qtpy : None
pyqt5 : None

[pauluhlenbruck]

take

[pauluhlenbruck]

I will start working on this issue during the upcoming weekend.

[pauluhlenbruck]

While thinking about the expected behaviour, I came across the test test_html.test_extract_links(), which resulted from #13141.

<table>
   <tr>
      <th>HTTP</th>
      <th>FTP</th>
      <th><a href="https://en.wiktionary.org/wiki/linkless">Linkless</a></th>
   </tr>
   <tr>
      <td><a href="https://en.wikipedia.org/">Wikipedia</a></td>
      <td>SURROUNDING <a href="ftp://ftp.us.debian.org/">Debian</a> TEXT</td>
      <td>Linkless</td>
   </tr>
   <tfoot>
      <tr>
         <td><a href="https://en.wikipedia.org/wiki/Page_footer">Footer</a></td>
         <td>
            Multiple <a href="1">links:</a> <a href="2">Only first captured.</a>
         </td>
      </tr>
   </tfoot>
</table>

HTTP	FTP	Linkless
Wikipedia	SURROUNDING Debian TEXT	Linkless
Footer	Multiple links: Only first captured.

Part of expected output for test_html.test_extract_links():

"footer_extract": [
                ("Footer", "https://en.wikipedia.org/wiki/Page_footer"),
                ("Multiple links: Only first captured.", "1"),
                None,
            ]

So it seems like only the first link being extracted from a cell is not a bug.
Changing this behavior would probably break backwards compatibility and seems more like enhancement of pandas.read_html().
Maybe others can give their opinion on how to proceed from here?

[kyoungrok0517]

Maybe we can have an optional argument to give 'extract all' direction. Should be default to False.