使用Selenium抓取TripAdvisor用户评论#
tripadvisor中文网站叫做猫途鹰 www.tripadvisor.cn/ 。
对于酒店和景点的用户评论。
但是默认只显示中文评论！
需要点击所有评论才能显示其它语言的用户评论。
url = 'https://www.tripadvisor.cn/Hotel_Review-g294213-d1459920-Reviews-Somerset_Jiefangbei_Chongqing-Chongqing.html#REVIEWS'
print(url)
https://www.tripadvisor.cn/Hotel_Review-g294213-d1459920-Reviews-Somerset_Jiefangbei_Chongqing-Chongqing.html#REVIEWS
from selenium import webdriver
browser = webdriver.Chrome()
url = 'https://www.tripadvisor.cn/Hotel_Review-g294213-d1459920-Reviews-Somerset_Jiefangbei_Chongqing-Chongqing.html#REVIEWS'
browser.get(url)
loc = '#component_10 > div > div:nth-child(3) > div.location-review-filters-hr-ReviewFilters__filters_wrap--3zsVa > div.ui_columns > div.ui_column.is-3-tablet.is-shown-at-tablet > ul > li:nth-child(1) > label > span.location-review-review-list-parts-LanguageFilter__no_wrap--2Dckv'
browser.find_element_by_css_selector(loc).click()
source = browser.page_source
soup = BeautifulSoup(source, 'html.parser')
comments = soup.find_all('div', {'class':'hotels-community-tab-common-Card__card--ihfZB hotels-community-tab-common-Card__section--4r93H'})
comments[0]
<div class="hotels-community-tab-common-Card__card--ihfZB hotels-community-tab-common-Card__section--4r93H" data-test-target="HR_CC_CARD"><div class="location-review-review-list-parts-MachineTranslationHeader__mt_wrap--2uRXX"><span class="location-review-review-list-parts-MachineTranslationHeader__qtext--2lhyR">这些点评为英文的机器翻译。 显示机器翻译？</span> <div class="ui_radio"><input checked="" id="autoTranslateYes" type="radio" value="true"/><label for="autoTranslateYes">是</label></div><div class="ui_radio"><input id="autoTranslateNo" type="radio" value="false"/><label for="autoTranslateNo">否</label></div><span class="ui_icon information-circle-fill location-review-review-list-parts-MachineTranslationHeader__tip--2EQEh"></span></div><div class="social-member-event-MemberEventOnObjectBlock__member_event_block--1Kusx"><a class="styleguide-avatar-Avatar__avatar--2NStU ui_social_avatar inline" href="https://www.tripadvisor.com/Profile/kimlette" target="_self"><img src="https://ccm.ddcdn.com/ext/photo-t/01/2e/70/81/avatar002.jpg"/></a><div class="social-member-event-MemberEventOnObjectBlock__event_wrap--1YkeG"><div class="social-member-event-MemberEventOnObjectBlock__member_actions--2uzIM"><div class="overflow"><span class="_3-zQ1eyd _4nIC4qqF"><span class="_35pfga6W _2HBN-k68"></span></span></div><span class="social-section-overflow-SectionOverflowMenu__float--3mJ5J"></span></div><div class="social-member-event-MemberEventOnObjectBlock__event_type--3njyv"><span><a class="ui_header_link social-member-event-MemberEventOnObjectBlock__member--35-jC" href="https://www.tripadvisor.com/Profile/kimlette">kimlette</a> 写了一条点评 12月12日 </span></div><div class="social-member-MemberHeaderStats__event_info--30wFs"><span class="social-member-MemberHeaderStats__hometown_stat_item--231iN"><span class="default social-member-common-MemberHometown__hometown--3kM9S small"><span class="ui_icon map-pin-fill social-member-common-MemberHometown__icon--3WpeC"></span>新加坡新加坡</span></span></div></div></div><div class="location-review-review-list-parts-SingleReview__mainCol--1hApa" data-reviewid="671457443"><div class="location-review-review-list-parts-RatingLine__container--2bjtw"><div class="location-review-review-list-parts-RatingLine__bubbles--GcJvM" data-test-target="review-rating"><span class="ui_bubble_rating bubble_10"></span></div></div><div class="location-review-review-list-parts-ReviewTitle__reviewTitle--2GO9Z" data-test-target="review-title" dir="ltr"><a class="location-review-review-list-parts-ReviewTitle__reviewTitleText--2tFRT" href="/ShowUserReviews-g294213-d1459920-r671457443-Somerset_Jiefangbei_Chongqing-Chongqing.html"><span lang="zh-x-mtfrom-en"><span>不要留在这里：在线图片欺骗</span></span></a></div><div class="location-review-review-list-parts-ExpandableReview__containerStyles--1G0AE"><div class="_2f_ruteS _1bona3Pu _2uD5bLZZ"><div class="cPQsENeY" style="max-height: initial;"><q class="location-review-review-list-parts-ExpandableReview__reviewText--gOmRC" lang="zh-x-mtfrom-en"><span>入住3晚，但在12月第一周晚1晚后退房。 Somerset ppty真是太可惜了，房间简朴而舒适。我们已经预订了3间客房，所以我们没有。房间布局中，只有一个可行的是09堆栈。所有级别只有一个09型单位。布局不好的是03面，所以如果不幸的话避免呆在这里。大堂和餐厅/餐厅的日子已经好过了，管理层需要尽快对其进行翻新。早餐太可悲了。我们有了早餐券，但是晚上睡不好后就没有吃。第二天早上尽快退房。不要留在这里！在公共区域也有难闻的气味。 唯一的好处是-接待女士们确实很有帮助和理解。在我们急忙离开的时候，我忘了问他们的名字了！</span></q></div><span class="_36B4Vw6t" style="line-height: 20px;"><div class="location-review-review-list-parts-ExpandableReview__ctaLine--24Qlb" data-test-target="expand-review"><span class="location-review-review-list-parts-ExpandableReview__cta--2mR2g">阅读更多</span><span class="ui_icon caret-down location-review-review-list-parts-ExpandableReview__caret--3Ud_i"></span></div></span></div><div class="hotels-community-tab-reviews-ReviewsTabContent__footer--2FHIK"><span class="location-review-review-list-parts-EventDate__event_date--1epHa"><span class="location-review-review-list-parts-EventDate__bold--3b3PE">入住日期：</span>2019年12月</span><span class="hotels-community-tab-reviews-ReviewsTabContent__hiddenReviewContent--FCrPr"><span class="location-review-review-list-parts-TripType__trip_type--3w17i"><span class="trip_type_label">旅行类型： </span>结伴旅行</span></span><div class="hotels-rate-mt-footer-rate-mt-footer__mtFooter--vvI3I"><span class="public-reviews-MTProviderLogo__provider_img--1z6u0"><img alt="Google" src="/img2/langs/zh/branding/partners/powered_by_google_translate.png"/></span><div class="hotels-rate-mt-footer-rate-mt-footer__inlinify--2w_D9"><div class="ui_button small ui_link standard hotels-rate-mt-footer-rate-mt-footer__rateButton--1mDkC">为译文打分 <span class="ui_icon caret-down"></span></div></div></div></div><div class="location-review-social-bar-SocialBar__bar--3YwG5"><span class="social-statistics-bar-SocialStatisticsBar__bar--L8RDK"></span><div class="social-actions-SocialActionBar__social_action_bar--2_j8Y"><span class="social-actions-SocialAction__social_action--1RrZ-"><span class="ui_icon thumbs-up-fill social-actions-SocialAction__icon--2WCF0"></span><span class="ui_link social-actions-SocialAction__text--1yJuk">推荐</span></span><span class="social-actions-NestedSocialAction__nested_social_action--2mhpU"><div><span><div class="public-share-cta-ShareCta__social_action--2yJtr"><span><span class="ui_icon repost-fill public-share-cta-ShareCta__icon--1Bvb0"></span><span class="ui_link public-share-cta-ShareCta__text--1tWKH socialCTA">分享 </span></span></div></span></div></span></div></div><div class="location-review-review-list-parts-OwnerResponse__containerStyles--1wv-S"><span class="styleguide-avatar-Avatar__avatar--2NStU ui_social_avatar small inline location-review-review-list-parts-OwnerResponse__ownerAvatar--3bb01"><span class="ui_icon member-fill"></span></span><div class="location-review-review-list-parts-OwnerResponse__header--2jRig">回复来自重庆盛捷解放碑服务公寓的前厅部经理SomersetChongqing<div class="location-review-review-list-parts-OwnerResponse__responseDate--4i8wE" title="2019-12-15">回复时间：3天前</div></div><div class="location-review-review-list-parts-GoogleTranslateButton__buttonDiv--3bQd_"><button class="ui_button secondary small">Google翻译</button></div><div class="_2f_ruteS _1bona3Pu location-review-review-list-parts-OwnerResponse__responseReadMore--eB9_V _2uD5bLZZ"><div class="cPQsENeY" style="max-height: 60px;"><span class="location-review-review-list-parts-OwnerResponse__reviewText--108-V"><span>I have noted your observations and recommendations for improvements to the breakfast and refurbishment. Please be assured that I will coordinate the plans for these enhancements with the respective departments. We seek your patience and understanding as the recommendations may require time to be implemented, we will continue in our improvement efforts to provide excellent service and products to our residents. </span></span></div><span class="_36B4Vw6t" style="line-height: 20px;"><span class="location-review-review-list-parts-OwnerResponse__cta--2QSGH">阅读更多</span><span class="ui_icon caret-down location-review-review-list-parts-OwnerResponse__caret--1-bUW"></span></span></div></div></div></div></div>
# user id
# https://www.tripadvisor.com/Profile/kimlette
comments[0].find('a', {'class', 'ui_header_link social-member-event-MemberEventOnObjectBlock__member--35-jC'}).text
'kimlette'
comments[0].find('span', {'class', 'default social-member-common-MemberHometown__hometown--3kM9S small'}).text
'新加坡新加坡'
# comment title
comments[0].find('a', {'class', 'location-review-review-list-parts-ReviewTitle__reviewTitleText--2tFRT'}).text
'不要留在这里：在线图片欺骗'
# comment body text
comments[0].find('q', {'class', 'location-review-review-list-parts-ExpandableReview__reviewText--gOmRC'}).text
'入住3晚，但在12月第一周晚1晚后退房。 Somerset ppty真是太可惜了，房间简朴而舒适。我们已经预订了3间客房，所以我们没有。房间布局中，只有一个可行的是09堆栈。所有级别只有一个09型单位。布局不好的是03面，所以如果不幸的话避免呆在这里。大堂和餐厅/餐厅的日子已经好过了，管理层需要尽快对其进行翻新。早餐太可悲了。我们有了早餐券，但是晚上睡不好后就没有吃。第二天早上尽快退房。不要留在这里！在公共区域也有难闻的气味。 唯一的好处是-接待女士们确实很有帮助和理解。在我们急忙离开的时候，我忘了问他们的名字了！'