Twitter Data API access in RSTUDIO

Get access to the Twitter API: https://developer.twitter.com/en/apply-for-access

Create an APP, as documented here: https://developer.twitter.com/en/apps

Annotation 2019-11-05 073528

Click on “Details”, then “Keys and tokens”

Annotation 2019-11-05 073800

(The credentials listed above will not work)

Use the generated credentials as below:

So where did the URL in GET come from?

The API Reference contains these instructions: https://developer.twitter.com/en/docs/tweets/timelines/api-reference/get-statuses-home_timeline

Annotation 2019-11-05 080500.png

MySQL Custom Install on Windows Startup Issue

OS: MS Windows 10

MySQL: Developer Computer configuration 8.0

Other: Custom path install, configured to NOT work as a service

Considerations: The software was recently installed, and ran properly when it was started through the installer. Nothing was changed on the installation of the software, but the host has been restarted

Issue: MySQL server will not start through the Workbench, and running

%MYSQL_HOME%\bin\mysqld.exe

will generate issue with missing data directory. If this data directory is not the correct %Program Data% specified, this means that the mysqld.exe does not have access to configuration file

my.ini

Solution: Place a copy of file my.ini in the root directory of Windows

(ie "C:\Windows").

Execute mysqld.exe once again and minimize its terminal

SOURCE: https://dev.mysql.com/doc/refman/5.5/en/windows-troubleshooting.html

 

xmlTreeParse Fails with error “XML content does not seem to be XML”

Code below is run in RSTUDIO.

  • fileURL is the variable that holds the internet location of the XML file in question
  • doc is the variable to store the output of xmlTreeParse

> fileUrl doc <- xmlTreeParse(fileUrl,useInternal=TRUE)
Error: XML content does not seem to be XML: ‘https://www.w3schools.com/xml/simple.xml&#8217;

SOLUTION: Install package RCurl and use function getURL on the link to the XML file  first

install.packages(“RCurl”)
WARNING: Rtools is required to build R packages but is not currently installed. Please download and install the appropriate version of Rtools before proceeding:

https://cran.rstudio.com/bin/windows/Rtools/
also installing the dependency ‘bitops’

trying URL ‘https://cran.rstudio.com/bin/windows/contrib/3.5/bitops_1.0-6.zip&#8217;
Content type ‘application/zip’ length 38894 bytes (37 KB)
downloaded 37 KB

trying URL ‘https://cran.rstudio.com/bin/windows/contrib/3.5/RCurl_1.95-4.12.zip&#8217;
Content type ‘application/zip’ length 2968691 bytes (2.8 MB)
downloaded 2.8 MB
> library(RCurl)
Loading required package: bitops
> xData <- getURL (fileUrl)
> doc <- xmlTreeParse(xData)
> doc
$doc
$file
[1]

[HOWTO] 100% CPU on Windows 10 Home, MSI Gaming Laptop (GP62MVR 7RFX Leopard PRO)

Read just this:

If you have already reset the laptop’s power plan settings, and have set a AUTO fan speed within MSI Dragon Center, probably nothing else needs be done.

Description of problem:

Windows Task Manager shows CPU to be 100% used, most of the load is by a Windows process. The issue occurs on an MSI laptop, which has MSI own management software, such as System Control Manager (SCM) and/or MSI Dragon Center.

Analysis:

MSI’s own applications interact with MS Windows power plans to set up the operating parameters of the system: Fan(s) speed counted in rpm, settings for battery life vs performance, timeouts. WIN10 will also switch power plans when the laptop is getting plugged/unplagged from its charger, which also alters the behavior of the fans. MSI Dragon Center and Windows power management are very tightly knit. Also there seems to be a connection between the power plan and CPU consumption. The more performance oriented is the power plan, the higher the CPU consumption will be. The main finding is that the CPU 100% does not relate to any specific process, but the power plan in effect.

Ways to not see 100% in Task Manager:

Step 1, this is how Task Manager looks now:

CPU100

…and this is how SCM looks, with Dragon Center installed:

scmwithdragon

 

Step 2: Uninstall MSI Dragon Center

…and this is how SCM looks now:

scmwithoutdragon

Notice how the section with the different “modes” appeared. Now let’s select any mode other than “ECO off”, and get a screenshot of MS Task Manager

 

scmwithoutdragonnoECOoff

 

CPU005

…quite a difference. There is an immediate, notable difference with the fans speed, the laptop becomes immediately quiet. What else has happened?

Right-click on the Windows logo on Your screen, select “Power Options” and at the following screen “Additional power settings”. There, is a new power plan, created by SCM, and named after the selected “mode”, for example mine is called “Gaming”. I tried to compare the detailed configuration of this plan (“Change plan settings”) with the one that is causing the CPU to peak, and i was unable to find the exact parameter that creates the whole issue. In fact, the new power plan is quite “aggressive” in some of its settings (acccessed by “Change plan settings” under Control Panel > Hardware and Sound > Power Options > Edit Plan Settings )

SCMPowerPlan

So what now?

With MSI Dragon Center installed again, there is no way to create further SCM power plans. Using the Windows default ones will revert to 100% CPU, but look:

dragoncentervstaskmanager

 

What is correct?

If anybody knows, and feels confident of their Hardware/Windows skills, please leave a comment. For myself, i have decided to leave a less restricted plan for the fans, accepting to see the CPU 100% message, which i somehow thing is not crucial. This, because i am worried that trying to demote the cooling system might affect the well-being of the motherboard of my Leopard. I suppose, the choice is yours.

How to exit Debug Browser in R

This works if you have enabled debugging for a single command/function, and you are not aware how to exit the Browser prompt. To save you the trouble, popular Googles that ask you to f or Q or ESC or q() simply do not work. The behavior of debug command is identical in RStudio and the R prompt, at least in Windows 10 that i tested.

Here are two examples of enabling/disabling debug.

First we use a random sapply command:

> sapply(split(mtcars$mpg, mtcars$cyl), mean)
4 6 8
26.66364 19.74286 15.10000

Now we turn debugging on:

> debug(sapply)

and, as expected, nothing happens until we try to run sapply again:

> sapply(split(mtcars$mpg, mtcars$cyl), mean)
debugging in: sapply(split(mtcars$mpg, mtcars$cyl), mean)
debug: {
FUN <- match.fun(FUN)
answer <- lapply(X = X, FUN = FUN, ...)
if (USE.NAMES && is.character(X) && is.null(names(answer)))
names(answer) <- X if (!isFALSE(simplify) && length(answer)) simplify2array(answer, higher = (simplify == "array")) else answer } Browse[2]>

There are two things to type here. The first is the command undebug(). In this case will be sapply. Then, Q exits the Browser prompt, and we can run the function outside of debug mode again. Let’s see:

Browse[2]> undebug(sapply)
Browse[2]> Q
> sapply(split(mtcars$mpg, mtcars$cyl), mean)
4 6 8
26.66364 19.74286 15.10000

Identically with ls()

ls(mtcars)
[1] "am" "carb" "cyl" "disp" "drat" "gear" "hp" "mpg" "qsec" "vs"
[11] "wt"
> debug(ls)
> ls(mtcars)
debugging in: ls(mtcars)
debug: {
if (!missing(name)) {
pos <- tryCatch(name, error = function(e) e)
if (inherits(pos, "error")) {
name <- substitute(name)
if (!is.character(name))
name <- deparse(name)
warning(gettextf("%s converted to character string",
sQuote(name)), domain = NA)
pos <- name
}
}
all.names <- .Internal(ls(envir, all.names, sorted))
if (!missing(pattern)) {
if ((ll <- length(grep("[", pattern, fixed = TRUE))) &&
ll != length(grep("]", pattern, fixed = TRUE))) {
if (pattern == "[") {
pattern <- "\\["
warning("replaced regular expression pattern '[' by '\\\\['")
}
else if (length(grep("[^\\\\]\\[<-", pattern))) {
pattern <- sub("\\[<-", "\\\\\\[<-",
pattern)
warning("replaced '[<-' by '\\\\[<-' in regular expression pattern") } } grep(pattern, all.names, value = TRUE) } else all.names }

Browse[2]>undebug(ls)
Browse[2]> Q
> ls(mtcars)
[1] "am" "carb" "cyl" "disp" "drat" "gear" "hp" "mpg" "qsec" "vs"
[11] "wt"
>

Linear Algebra in R, create and invert matrices

This is a simple test case of creating a random 2×2 matrix, performing its inversion, and multiplying them. We will also use MS Excel to check our computations
Create a 2×2 matrix

> mNP <- matrix(rnorm(4),nrow=2,ncol=2)

This command used the matrix() function . This first argument, the datasets, uses rnorm() function to generate 4 random variates, and the next two arguments explain that those are to be ordered in a matrix of 2 rows by 2 columns. Now let’s display the matrix

> mNP
[,1] [,2]
[1,] 0.5644179 -0.4694577
[2,] 0.7707571 0.1500823

Next, we use solve() function to invert the matrix, and display the output

> mNP_inv <- solve(mNP)
> mNP_inv
[,1] [,2]
[1,] 0.3360953 1.051306
[2,] -1.7260381 1.263961

Finally, we use the %*% operator to to the algebraic multiplication of the matrices and check if we arrive to I

> mNP %*% mNP_inv
[,1] [,2]
[1,] 1 0
[2,] 0 1
>

If you would want to do the last operation in Excel, as a check, then let’s suppose that the first matrix sits in fields A1,B1,A2,B2 and the second one sits in fields D1,E1,D2,E2. Their multiplication product would be:
Top left, top right
=A1*D1+B1*D2 =A1*E1+B1*E2
Bottom left, bottom right
=A2*D1+B2*D2 =A2*E1+B2*E2

Make a bar-chart from a CSV in Python

plot.ly CSV bar plot
Plot.ly bar chart using a CSV

Test Case was implemented in Python 3.6.5 running on a Ubuntu Linux 18.04 64-bit virtual machine. In order to carry out this test-case you will need to create an account in plot.ly and create the credentials file on the host you will be running Python from. All instructions are on their web site

Step 1:
Suppose a CSV which has a first row we want to define as the X-Axis of our plot, and two further rows which we want as the data in the Y-Axis. It could be something like this:

~$ cat /home/nikolas/categories.csv
SciFi-Fantasy , 31.550787 , 68.449219
Spirituality , 83.411890 , 16.588112
Home-Improvement , 47.082787 , 52.917217
Gaming , 2.256584 , 97.743423
Mountain-Bike-Touring , 40.905171 , 59.094826
Korean-Culture , 71.040140 , 28.959862
Health-Safety , 32.872467 , 67.127533
Religion , 37.452973 , 62.547028
Fashion , 98.597282 , 1.402729

Step 2:
Load the CSV into a data frame using library Pandas using function read_csv and display the data of each row, using function iloc:

Python 3.6.5 (default, Apr 1 2018, 05:46:30)
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas as pd
>>> mycatg=pd.read_csv('/home/nikolas/categories.csv',sep=',',header=None)
>>> mycatg.iloc[:,0]
0 SciFi-Fantasy
1 Spirituality
2 Home-Improvement
3 Gaming
4 Mountain-Bike-Touring
5 Korean-Culture
6 Health-Safety
7 Religion
8 Fashion
Name: 0, dtype: object
>>> mycatg.iloc[:,1]
0 31.550787
1 83.411890
2 47.082787
3 2.256584
4 40.905171
5 71.040140
6 32.872467
7 37.452973
8 98.597282
Name: 1, dtype: float64
>>> mycatg.iloc[:,2]
0 68.449219
1 16.588112
2 52.917217
3 97.743423
4 59.094826
5 28.959862
6 67.127533
7 62.547028
8 1.402729
Name: 2, dtype: float64

Step 3:

Include the plot.ly libraries

import plotly.plotly as py
import plotly.graph_objs as go

then define the axis data, using the above way to address the rows in the data frame. so:

yaxis1 = go.Bar(
x=myres.iloc[:,0],
y=myres.iloc[:,1],
name='Category A'
)
yaxis2 = go.Bar(
x=myres.iloc[:,0],
y=myres.iloc[:,2],
name='Category B'
)

data = [yaxis1, yaxis2]
layout = go.Layout(
barmode=’simple’
)

then perform the plot itself.

fig = go.Figure(data=data, layout=layout)
py.iplot(fig, filename='barplot in Plot.ly, smalldeskbigdata.com')

Step 4:

This is it. The plot is created as a SVG file in plot.ly. In this case the graph was created here:

https://plot.ly/create/?fid=NotNikolas63500%3A12