Sentiment
Analysis of User Reviews of E-commerce Applications: Case Study on the Shoppe
Platform
Aisah Rini Susanti1*, Elita Nur Ilahi2�����
Computer Science Study Program, Faculty
of Computer Science, Universitas Djuanda, Indonesia1*2
Email: [email protected]1*
ABSTRACT
The
development of e-commerce in Indonesia is driven by the large population and
extensive geography. One of the leading e-commerce applications in Indonesia is
Shopee. User reviews of this application reflect public sentiment towards
Shopee. The Naive Bayes method was used to classify reviews, with an accuracy
rate of 90.76%. Using TF-IDF helps calculate the weight of words in reviews.
Performance evaluation shows that the model has high accuracy in scenario 1
with a 60:40 split between training data and test data. The use of information
technology has changed the business paradigm, including in e-commerce such as
Shopee. The importance of understanding consumer sentiment can be seen from
social media platforms, where sentiment analysis using Naive Bayes and Topic
Modeling shows negative sentiment at the 11:11 Shopee Flashsale event.
Suggestions for Shopee include improving event strategies such as Flashsale by
inviting artists and improving customer service. Responsiveness to technical
application problems is also emphasized to increase customer satisfaction. The
increase in e-commerce transactions in Indonesia shows rapid growth, with many
platforms emerging. Sentiment analysis such as that done with K-Nearest Neighbor
(K-NN) on Google Play Store reviews provides insight into the experience of
e-commerce users in Indonesia, with an accuracy rate of up to 82%. This
research provides significant implications for e-commerce companies in better
understanding and meeting customer expectations.
Keywords: e-commerce, shoppe, google play store,
sentiment.
INTRODUCTION
In
Indonesia, the use of e-commerce is growing rapidly due to a shift in consumer
behavior which is increasingly shopping online (Alfanur & Kadono, 2021). Based on the latest
report from We Are Social, around 178.9 million Indonesians have shopped online
from the beginning of 2022 to the end of 2023, recording growth of 12.8% every
year. Apart from that, it is estimated that the Indonesian population's online
spending reached US$55.97 million or around Rp. 851 trillion in the same
period.
However,
maintaining the user base and improving the service quality of an e-commerce
platform relies heavily on customer satisfaction (Ali et al., 2021). Several e-commerce platforms that can be used as references include
Shopee, Tokopedia, Lazada, Bukalapak, and Blibli (Putri et al., 2023). Shopee was chosen as the research object in analyzing public
sentiment because of its popularity which dominates the e-commerce market in
Indonesia (Salim et al., 2021). To understand the level of user satisfaction and trust in this
platform, it is necessary to analyze public sentiment regarding its use (Ye et al., 2023).
This
research proposes an approach to conduct sentiment analysis of public opinion
regarding the use of Shopee using the K-Nearest Neighbor algorithm (Kirana & Al Faraby, 2021). Data was collected from
the Google Play Store website using web scraping techniques, which were then
processed by removing stopwords, tokenization, and stemming before applying the
K-Nearest Neighbor (K-NN) and Natural Language Processing (NLP) algorithms to
classify sentiment into positive or negative. Evaluation is carried out using a
confusion matrix and classification report (Tharwat, 2021).
The
development of e-commerce makes transactions between sellers and buyers easier
through smartphone applications that allow easy access anytime and anywhere.
According to data from APJII, e-commerce transactions in Indonesia increase
every year, showing significant growth in the use of e-commerce. However, with
so many e-commerce emerging, people are often confused about choosing a trusted
platform. Therefore, sentiment analysis on social media is needed to determine
the level of popularity of e-commerce based on positive or negative opinions
expressed by the public.
RESEARCH
METHODS
In this
research, Twitter is used as social media to collect public opinion because of
its popularity in conveying opinions directly. Twitter allows interaction
between users that helps in decision making. Sentiment analysis was carried out
using the Na�ve Bayes method, a machine learning technique for classifying text
data based on opinion classes. Na�ve Bayes improves scalability, accuracy, and
efficiency in the text classification process.
Based on
this problem, this research analyzes the level of popularity of e-commerce in
Indonesia using sentiment analysis on Twitter social media with the Na�ve Bayes
method. The development of e-commerce technology in Indonesia facilitates
digital transactions between organizations and individuals. The value of
e-commerce transactions increased from IDR 25 trillion in 2014 to IDR 69.8
trillion in 2016, and is projected to reach IDR 114 trillion in 2018.
The growth
in social media use also supports the increase in digital purchases of goods.
Shopee, as the e-commerce with the largest number of followers on Instagram in
Indonesia, also occupies the top ranking of shopping applications on iOS and
Android. According to the 2018 APJII survey, internet users in Indonesia
increased to 143.26 million people or 54.7% of the total population.
In
business, effective marketing strategies, such as 11:11 sales by Shopee, can
increase product sales. Data from social media called User Generated Content
(UGC) contains public opinion which can be analyzed to understand consumer
sentiment. Sentiment analysis is a scientific process for classifying text
based on opinions in the text, determining whether the text has positive, negative,
or neutral meaning. Sentiment analysis is also known as opinion mining because
it originates from a person's opinions or attitudes.
RESULTS
AND DISCUSSION
A. Data Description
The
dataset used in this research comes from user reviews of the Shopee application
taken from the Google Play Store using web scraping techniques with the help of
WebHarvy software (Bustami & Noviaristanti, 2022). This dataset contains 895
rows of data with Review and Sentiment columns (Ngo et al., 2022).
B. Data
Preprocessing
The data
preprocessing stage involves data cleaning processes such as changing text to
lowercase, removing punctuation marks, and deleting unnecessary characters (Abidin et al., 2019). After that, the sentence is broken down into tokens or words
through a tokenization process (Vijayarani & Janani, 2016). This process is followed
by stopword removal and stemming to obtain relevant root words (Kalaivani & Marivendan, 2021).
C. Sentiment
Analysis Model
The
sentiment analysis model in this research uses the NLP and KNN algorithms (Demircan et al., 2021). The model training process is divided into two parts: training
set and testing set with a proportion of 80:20 (Shu et al., 2020). The TF-IDF technique is used to calculate the weight of each
word in the review. In the training stage, the KNN algorithm is used to
classify each document into positive or negative sentiment classes (Isnain et al., 2021). This model managed to achieve an accuracy of 82% and a cross
validation score of 80%, indicating that the combination of the NLP and KNN
algorithms is effective in sentiment analysis (Dake & Gyimah, 2023).
D. Model
Evaluation
Model
evaluation was carried out using confusion matrix and cross validation score.
The confusion matrix is used to show the number of correct and incorrect
classifications of the model, while the cross validation score measures how
well the model predicts data that has never been seen before.
Visualization
Data
visualization was carried out using wordcloud to analyze the frequency of the
most dominant words in user reviews. This visualization makes it easier to
understand user preferences and experiences regarding e-commerce usage, and
helps identify the most significant keywords in sentiment analysis.
Data
collection
The text
data used in this research comes from tweets and retweets by e-commerce
customers such as Lazada, Tokopedia, Bukalapak, and Shopee. For each
e-commerce, 500 tweet data was taken which was limited to one week before the
collection date according to Twitter rules.
Preprocessing
The
preprocessing process includes cleaning unnecessary characters such as URL, @,
#, and other symbols, followed by case folding to convert all characters to
lowercase.
Na�ve Bayes Classification
The
Naive Bayes classification process is carried out using the Rapidminer
application. The preprocessed data is entered into the application using steps
such as entering a labeled dataset, selecting attributes as primary keys,
entering training data, and carrying out Na�ve Bayes classification. The
results of the Naive Bayes classification are displayed in a diagram to
determine the level of popularity of e-commerce in Indonesia based on sentiment
on Twitter social media.
Testing
The test
aims to determine the accuracy of the Na�ve Bayes classification. This test
uses the Performance method in the RapidMiner application. The test results
show the percentage of negative and positive e-commerce sentiment.
LDA-Based
Topic Modeling Results
By using
LDA-based topic modeling processed using R-Studio, the data was analyzed to see
the topics discussed by users in the Shopee Instagram comments column. This
analysis identified six main topics during the period 25 October 2018 to 11
November 2018, including complaints about application errors, disappointment
with flash sales, the length of the shopping process, disappointment with
quizzes, and dissatisfaction with cashback and free shipping programs.
On the
positive side, the five main topics identified include enthusiasm for Shopee
events, discounts that are considered good, support for Shopee despite
obstacles, and recognition that Shopee is the best marketplace.
Negative
topics include disappointment with a slow network, promotions and cashback that
are considered unsatisfactory, server problems that often experience errors,
slow Shopee admin responses, and frustration with flash sales and quizzes that
are considered unfair.
Figure 2. 1Positive
LDA-Based Topic Modeling Results
Figure 1 1Negative
LDA-Based Topic Modeling Results
Figure 3. 1Results of
LDA-Based Topic Modeling
CONCLUSION
This research
employs the K-Nearest Neighbor (K-NN) algorithm and Natural Language Processing
(NLP) techniques to analyze user sentiment towards the Shopee e-commerce
application. The dataset comprises 895 user reviews obtained through web
scraping from the Google Play Store. Preprocessing steps involved converting
text to lowercase, removing punctuation, and eliminating irrelevant characters,
followed by tokenization, stopword removal, and stemming to standardize the
text. The sentiment analysis model combines NLP techniques with K-NN and utilizes
TF-IDF for word weighting, achieving 82% accuracy and an 80% cross-validation
score in classifying reviews as positive or negative. Model evaluation using a
confusion matrix and cross-validation demonstrates its robustness in predicting
sentiment on unseen data. Visualization techniques, including word clouds,
highlight frequently mentioned words in user reviews, providing deeper insights
into preferences and experiences. Additionally, LDA-based topic modeling
identifies key topics such as application errors, flash sale disappointments,
and enthusiasm for cashback events. The study concludes that Shopee's
popularity and mixed user sentiment, encompassing both complaints and
enthusiasm for promotions, underscore the effectiveness of the NLP and K-NN approach
in sentiment analysis. This research contributes valuable insights for
e-commerce companies, particularly Shopee, in enhancing service quality and
customer satisfaction based on user feedback.
REFERENCE
Abidin, D. Z., Nurmaini, S., Malik, R. F., Rasywir, E., &
Pratama, Y. (2019). A model of preprocessing for social media data extraction. 2019
International Conference on Informatics, Multimedia, Cyber and Information
System (ICIMCIS), 67�72.
Alfanur, F., & Kadono, Y. (2021). Empirical Study Of Purchase
Intention And Behavior Of E-Commerce Consumers In Indonesia. Malaysian E
Commerce Journal, 5(1), 20�28.
Ali, B. J., Saleh, P. F., Akoi, S., Abdulrahman, A. A., Muhamed, A. S.,
Noori, H. N., & Anwar, G. (2021). Impact of service quality on the customer
satisfaction: Case study at online meeting platforms. Ali, BJ, Saleh, Akoi,
S., Abdulrahman, AA, Muhamed, AS, Noori, HN, Anwar, G.(2021). Impact of Service
Quality on the Customer Satisfaction: Case Study at Online Meeting Platforms.
International Journal of Engineering, Business and Management, 5(2),
65�77.
Bustami, D. K., & Noviaristanti, S. (2022). Service quality analysis
of tokopedia application using text mining method. International Journal of
Management, Finance and Accounting, 3(1), 1�21.
Dake, D. K., & Gyimah, E. (2023). Using sentiment analysis to evaluate
qualitative students� responses. Education and Information Technologies,
28(4), 4629�4647.
Demircan, M., Seller, A., Abut, F., & Akay, M. F. (2021). Developing
Turkish sentiment analysis models using machine learning and e-commerce data. International
Journal of Cognitive Computing in Engineering, 2, 202�207.
Isnain, A. R., Supriyanto, J., & Kharisma, M. P. (2021).
Implementation of K-Nearest Neighbor (K-NN) algorithm for public sentiment
analysis of online learning. IJCCS (Indonesian Journal of Computing and
Cybernetics Systems), 15(2), 121�130.
Kalaivani, E. R., & Marivendan, E. R. (2021). The effect of stop word
removal and stemming in datapreprocessing. Annals of the Romanian Society
for Cell Biology, 25(6), 739�746.
Kirana, Y. D., & Al Faraby, S. (2021). Sentiment analysis of beauty
product reviews using the K-nearest neighbor (KNN) and TF-IDF methods with
chi-square feature selection. Journal of Data Science and Its Applications,
4(1), 31�42.
Ngo, A., Candri, A., Ferdinan, T., Kocoń, J., & Korczynski, W.
(2022). Studemo: A non-aggregated review dataset for personalized emotion
recognition. Proceedings of the 1st Workshop on Perspectivist Approaches to
NLP@ LREC2022, 46�55.
Putri, L. T., Aini, A. P. N., Solekah, N. A., & Istiqomah, D. F.
(2023). Exploring e-commerce adoption in Small and Medium Enterprises (SMEs)
using the technology acceptance model. International Journal of Business Economics
(IJBE), 5(1), 15�30.
Salim, M., Oktaviany, P., & Hayu, R. S. (2021). Study of Online
Impulsive Buying in Indonesia During Covid-19 Pandemic Outbreak. APMBA (Asia
Pacific Management and Business Application), 10(2), 125�136.
Shu, X., Zhang, L., Wang, Z., Lv, Q., & Yi, Z. (2020). Deep neural
networks with region-based pooling structures for mammographic image
classification. IEEE Transactions on Medical Imaging, 39(6), 2246�2255.
Tharwat, A. (2021). Classification assessment methods. Applied
Computing and Informatics, 17(1), 168�192.
Vijayarani, S., & Janani, R. (2016). Text mining: open source
tokenization tools-an analysis. Advanced Computational Intelligence: An
International Journal (ACII), 3(1), 37�47.
Ye, X., Su, X., Yao, Z., Dong, L., Lin, Q., & Yu, S. (2023). How Do
Citizens View Digital Government Services? Study on Digital Government Service
Quality Based on Citizen Feedback. Mathematics, 11(14), 3122.
|
Copyright
holder: Aisah Rini Susanti, Elita Nur Ilahi (2024) |
|
First publication right: |
|
This article is licensed under:
|
