Linux Directory Structure
Take a look at the File Manager shown below. Each folder serves a defined
purpose, most importantly the Security. Linux is typically safer than Microsoft
Windows just because of the way Linux works within these system directories.
Let’s take a look at each one of these directories.1. Root /
Root directory is the beginning point for the file system hierarchy. Every single file and directory starts from the root directory. As the name suggests, only the root user has write privilege under this directory. But it’s important to note that “/” is not related to “/root” in anyway. All the below listed directories reside in the Root Directory.
2.
User Binaries /bin
The “bin” directory is home to all the executable programs.
In Linux, the executables are termed as “Binaries”. All commands used by the
users of the computer are located here.
3. System Binaries s/bin
s/bin directory also contains binary commands just like /bin
directory. The only difference is the Linux commands located under this
directory are used by system administrator. Obviously system admin uses it for
system maintenance purpose
4. Boot Loader /boot
As the name of the folder indicates, /boot folder contains
boot loader related files including the important kernel, initrd, vmlinux, grub
files, etc..
5. System configuration files /etc
This is the the place where all the configuration files
required by all programs are stored. Each program will have a config file
that is used to control the operation of a program including start or stop of
the program. It is not an executable binary.
6. Home /home
Home directory has all users personal files. Inside it, each user account of the system will have a individual directory for instance “Tahir” in my system.7. System Libraries /lib /lib64
8. Devices /dev
/dev contains the files that are used by the USB devices attached to the computer.9. Mount directory /mount
This is a temporary mount directory which is typically used by the system administrators for mounting the different file systems.10. Optional add-on Apps /opt
/opt is for the installation of add-on application software packages. It is typically used for third-party software, which implies they are already pre-compiled.11. Process Information /proc
The proc file system is a pseudo-file system which is used as an interface to kernel data structures. This file system stores text information about system resources.12. Root /root
When a user sign in as the root user, there will be a separate home directory created.13. Temporary space /tmp
Tmp directory is a temporary folder that will be periodically deleted. In fact, some Linux distributions cleanup during each boot. The directory is used by the system programs for temporarily storing the data. For example, when you extract an archive, the contents are first extracted to the /tmp directory, and are then moved to the location you provided.14. User folder /usr
This folder is home to all user related programs, libraries, and documentation. The sub-directories in /usr relate to those described in this article.15. Variable data /var
Variable data like system log files reside here. These files get constantly modified depending on ongoing user and system tasks.-------------------------------------------------------------------------------------------------------
ODBC Configuration Under System DSN in OS
Name Platform Driver
Deer 64bit Cloudera ODBC driver for Deer
ZER 64bit SQL somewhere 15
TDB 64bit TD Database ODBC Driver 15.00
---------------------------------------------------------------------------------------------------------------------
3 Methods to develop data frames in R and Query the data frames
--------------------------------------------------------------------------------------------------------------------
# Using no package
Path1=paste0("D:/folder/1.csv") #get the path of your csv file and data
output1<- read.csv(Path1, sep = "," , header = TRUE) # read the csv file and your dataframe is ready
print(head(output1)) # print the top row of your dataframe
newdata <- subset(output1, output1$colname=="abc")
write.csv(newdata, file=paste("D:/folder/2.csv", sep=""), row.names = FALSE) # write output to the file
----------------------------------------------------------------------------------------------------
#Using dplyr:packages
library(dplyr)
Path1=paste0("D:/folder/1.csv") #get the path of your csv file and data
output1<- read.csv(Path1, sep = "," , header = TRUE) # read the csv file and your dataframe is ready
newoutput=filter(output1, output1$colname=="abc" & output1$colname2 >5) # apply filer on the data
print(head(newoutput))
#Using sqldf:packages
library(sqldf)
Path1=paste0("D:/folder/1.csv") #get the path of your csv file and data
output1<- read.csv(Path1, sep = "," , header = TRUE) # read the csv file and your dataframe is ready
newsqloutput=sqldf('SELECT date,anynumber FROM output1 WHERE anynumber="123456" and date='2021/07/12'') # apply the sql on the data
print(head(newsqloutput))
--------------------------------------------------------------------------------------------------------------------Python django web development Example
-----------------------------------------------------------------------------------------------------------manage.py= is command line utility.
__init__.py= is any empty directory consider as package
setting.py= consist of all the setting necessary for the project
urls.py= consist of all the urls used in this project.
wsgi.py = forward the request to the webapplication or webservers
step1.
directory\webproject>django-admin startproject DEMOPROJECT
This will create following directories in DEMOPROJECT
manage.py= is command line utility.
__init__.py= is any empty directory consider as package
setting.py= consist of all the setting necessary for the project
urls.py= consist of all the urls used in this project.
wsgi.py = forward the request to the webapplication or webservers
Step2.
directory\webproject\demoproject>run command
python manage.py runserver
step2. Ctr+c return to terminal
step 3
python manage.py startapp DEMOAPP
First change the views.py file
# Create your views here.
from django.http import HttpResponse
def hi(request):
return HttpResponse('<h1> This is my website</h1>')
step 4.
Creat a urls.py file in DemoApp and connect it to urls.py of DemoProject
urls.py file in DemoApp
------------------------
from django.urls import path
from . import views
urlpatterns = [path('', views.hi, name='home-page'),]
connect the DemoApp urls.py to urls.py file of Demo Project
urls.py file in DemoProject change for having connection with DemoApp
------------------------
from django.urls import path, include urlpatterns = [path('admin/', admin.site.urls), path('', include('DEMOAPP.urls'))]
Step 5.
Create a templates directory with DEMOAPP directory then within that template directory again create a
directory having same name as DEMOAPP then create a hi.html file in that DEMOAPP directory
step 6. Copy the name of class from the apps.py and go to DemoProject and go to setting.py file and write the name of the 'DemoAPP.apps.DemoappConfig' under Installed_APPS
# Application definition in setting.py file
INSTALLED_APPS = [
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'DEMOAPP.apps.DemoappConfig',
]
step 7.In Template under the demoapp go to views.py def the render function.
from django.shortcuts import render
from django.http import HttpResponse
def hi(request):
return render(request, 'DEMOAPP/hi.html')
=====================================================================
options(java.parameters = "-Xmx1000m")#increasejava heap size for reading big files#date shoud be in Y-M-D formate
#par(mfrow=c(2,2))
myfile=read.csv("2.csv",colClasses=c("Date"="Date"))
x=myfile$Date
y1=myfile$total
y2=myfile$chat
plot(x,y1,col= "red",xlab="DATES",ylab="Total", type="l")
par(new=TRUE)
plot(x, y2, type = "l", axes = FALSE, bty = "n", xlab = "", ylab = "")
#as.Date(myfile$Date,"%m/%d/%Y")
##funtion definition####
#elapsed_months <- function(end_date, start_date) {
#ed <- as.POSIXlt(end_date)
#sd <- as.POSIXlt(start_date)
#12 * (ed$year - sd$year) + (ed$mon - sd$mon)
#}
###function cal
##elapsed_months(Sys.Date(), "YYYY-01-01");
-----------------------------------------------------------------------------------------------------------------
library(xlsx)
# load file contents
file <- "1.xlsx"
wb <- loadWorkbook(file)
sheets <- getSheets(wb)
sheet <- sheets[[1]] # or another
# data to put into B10:H20 range
#data <- matrix(runif(7*11), nrow=11, ncol=7)
################################################################################################################
myfile=read.csv("2.csv",colClasses=c("Date"="Date"),header = TRUE)
x=as.Date(myfile$Date,"%Y-%m-%d")
y=myfile$col1
z=myfile$col2
data <- matrix(c(x[1],y[1],z[1]), nrow = 1, byrow = TRUE)
data[1,1] <- as.list(as.Date(x[1], format = "%Y-%m-%d"),1)
print(data[1])
#####################################################################################################################################3######
# modify contents
addDataFrame(data, sheet, col.names = FALSE, row.names = FALSE, startRow = 2, startColumn = 1)
# save to disk
saveWorkbook(wb, file)
==========================================================================
Writing to Excel file from CSV file and plot it mean while.
=============================================================
library(xlsx)
# load the excel file in which you want to wriet
file <- "1.xlsx"
wb <- loadWorkbook(file)
sheets <- getSheets(wb)
sheet <- sheets[[1]] # sheet in which you want to wirte the context of your data.
#################################################################################################################################################
#Creat a data frame contain your data using read.csv function or read.table or Convert array a2 into a data frame d2 = as.data.frame(a2)
#################################################################################################################################################
myfile=read.csv("2.csv",colClasses=c("Date"="Date"),header = TRUE)
x=as.Date(myfile$Date,"%Y-%m-%d")
y=myfile$Total
z=myfile$Snapchat
plot(x,y,col= "red",xlab="DATES",ylab="Total", type="l")
par(new=TRUE)
plot(x, z, type = "l", axes = FALSE, bty = "n", xlab = "", ylab = "")
for(i in 1:length(x)){ #for through the length of dataframe , we can take any column of dataframe that is myfile. here we took x column containing date.
# add data to excel sheet_1 via dataframe that is x,y,z
addDataFrame(x[i], sheet, col.names = FALSE, row.names = FALSE, startRow = i+1, startColumn = 1)
addDataFrame(y[i], sheet, col.names = FALSE, row.names = FALSE, startRow = i+1, startColumn = 2)
addDataFrame(z[i], sheet, col.names = FALSE, row.names = FALSE, startRow = i+1, startColumn = 3)
# save to disk
saveWorkbook(wb, file)
legend("topleft", legend=c("Line 1", "Line 2"),
col=c("red", "blue"), lty=1:1, cex=0.8,
title="Line types", text.font=4, bg='lightblue')
=======================================================================
using sql and vlookup in R on the data frame that the loaded data through .txt or csv files
===================================================================
hous <- read.table(
header = TRUE,
stringsAsFactors = FALSE,
text="HouseType HouseTypeNo
Semi 1
Single 2
Row 3
Single 2
Apartment 4
Apartment 4
Row 3"
) #end read table
---------------------------------------------------------------------------------------------------------------------------------------
largetable <- data.frame(HouseType = as.character(sample(unique(hous$HouseType), 1000, replace = TRUE)), stringsAsFactors = FALSE)
----------------------------------------------------------------------------------------------------------------------------------------------
lookup <- unique(hous)
-----------------------------------------------------------------------------------------------------------------------------------------------------
library(sqldf)
sqldf1 <- sqldf("SELECT largetable.HouseType, lookup.HouseTypeNo
FROM largetable
INNER JOIN lookup
ON largetable.HouseType = lookup.HouseType")
write.csv(sqldf1,file=paste("D:/test/House.csv",sep=""),row.names=FALSE)
======================================================================
Writing to Excel sheet in R Directly.
==================================================================
#directly save to existing excel file name 1.xlsx
file <- "1.xlsx"
wb <- loadWorkbook(file)
sheets <- getSheets(wb)
sheet <- sheets[[1]] # writing to sheet 1
#hous is a data fram
addDataFrame(hous, sheet, col.names = FALSE, row.names = FALSE, startRow = 2, startColumn = 1)
saveWorkbook(wb, file)
#save to disk in workbook
# Writing your data frame name hous to new excel sheet name 2.xlsx that will be created automatically
> hous <- read.table(
header = TRUE,
stringsAsFactors = FALSE,
text="HouseType HouseTypeNo
Semi 1
Single 2
Row 3
Single 2
Apartment 4
Apartment 4
Row 3"
) #end read table
> write.xlsx(hous, file = "2.xlsx",sheetName = "TestSheet", row.names = FALSE)
================================================================
connecting R to Excel
==========================================================
require(RODBC)
conn = odbcConnectExcel("myfile.xlsx") # open a connection to the Excel file
sqlTables(conn)$TABLE_NAME # show all sheets
df = sqlFetch(conn, "Sheet1") # read a sheet
df = sqlQuery(conn, "select * from [Sheet1 $]") # read a sheet (alternative SQL sintax)
close(conn) # close the connection to the file
----------------------------------------------------------------------------------------------------------------------
R vs Tableua
----------------------------------------------------------------------------------------------------------------------
STEP 1
run the following in r environment.
a library(Rserve);
b Rserve()
Starting Rserve...
"C:\PROGRA~1\R\R-33~1.1\library\Rserve\libs\x64\Rserve.exe"
STEP 2
Go to Tabluea Help Menu and then navigate to Setting and Performance and
then click Manager External Service Connection and click test connection
STEP 3
Pass Expressions to R
In order to let tableau know that the calculations need to go to R, it must be passed through one of the 4 functions.
These 4 functions are : SCRIPT_BOOL , SCRIPT_INT , SCRIPT_REAL , SCRIPT_STR
R Functions are computed as Table calculations in Tableau.
Since these are table calculations, all the Fields being passed to R must be aggregated like Sum(PROFI)
Passing variable to tableau from r script to Tableau with calculation and in reverse from Tableau to R
--To combine the file in one file using R
txt_files_ls =c("1.txt", "2.txt", "3.txt")
txt_files_df=lapply(txt_files_ls, function(x) {read.table(file = x, header = T, sep =",")})
combin <- do.call("rbind", lapply(txt_files_df, as.data.frame))
--to get the Unique values
library(sqldf)
df=read.csv(choose.files(),header =TRUE,sep ="," )
mydata=sqldf("select * from df")
unique(mydata)
---to get unique value using batch script
@echo off & setlocal EnableDelayedExpansion
set row=###
for /F "delims=" %%i in ('type "2.txt" ^| sort') do (
if /I not "%%i"=="!row!" (echo.%%i >> "File.new" & set row=%%i)
)
ren "2.txt" "File.old"
ren "File.new" "File.txt"
-------------------------------------------------------------------------------
for send email from outlook
---------------------------------------------------------------------
library (RDCOMClient)
OutApp <- COMCreate("Outlook.Application")
outMail = OutApp$CreateItem(0)
outMail[["To"]] = "myemail@hotmail.com"
outMail[["subject"]] = "Test R Language program"
outMail[["body"]] = "R programe to send email with attachment"
outMail[["attachments"]]$Add("c:/1.txt")
outMail$Send()
--------------------------------------------------------------------------------------
proceduresqlserver
create procedure insertbulk
@FileNamePath varchar(max)
AS
begin
declare @Bulk varchar(4000) = 'BULK INSERT MYTABLE FROM '''+@FileNamePath+''' WITH (FIELDTERMINATOR = '','', ROWTERMINATOR = ''\n'')'
exec (@Bulk)
end
EXEC insertbulk 'D:\data.csv'
--------------------------------------
integratewithR
----------------------------------------------
exec.store.proc <- function(param.name){
# load the RODBC package
require("RODBC");
# establish a connection
ch <- odbcConnect("sqlserver",uid="sa",pwd="****");
# construct a query string with a parameter
query <- paste("exec dbo.insertbulk '",param.name,"'",sep="");
# execute the query
df<-sqlQuery(ch, query);
# print the results
print(df);
}
exec.store.proc('D:/data.csv')
-------------------------
USE DB
declare @TableName varchar(20)
set @TableName='mytab'
declare @FilePathName varchar(20)
set @FilePathName='D:\directory\'
declare @FileName varchar(20)
set @FileName='test.txt'
EXEC('select * into ' + @TableName + ' from
OpenRowSet(''Microsoft.ACE.OLEDB.12.0'',''Text;Database='+@FilePathName+';'',''SELECT * From ' +
@FileName + ''')')
-- EXEC('select * into '+@TableName+'
from OpenRowset(''Microsoft.ACE.OLEDB.12.0'',''Text;Database='+@FilePathName+';'',''SELECT
*FROM '+@FileName+''')')
-- select
* into mytabltesting from
OpenRowSet('Microsoft.ACE.OLEDB.12.0',
'Text;Database=D:\directory\;','SELECT * from test.txt')
---------------------------------------------------
sp_configure 'show advanced options',1;
RECONFIGURE;
GO
sp_configure 'Ad Hoc Distributed
Queries',1;
--------------------------------------------------------library(xlsx)
setwd("D:/directory")
source("D:/directory/R_Function.r")
# load the excel file in which you want to wriet
file <- "yourinputfile.xlsx"
wb <- loadWorkbook(file)
sheets <- getSheets(wb)
sheet <- sheets[[1]]
rows=getRows(sheet)
cells=getCells(rows)
values <- lapply(cells, getCellValue) # extract the values
#for(i in 2:3){
#print(values[i][1])
exec.store.proc(values[3][1])
#}
------------------------------------------------------------------------------------------
Python ODBC code
def connection( sql,file_name ):
try:
pyodbc.autocommit = True
ch=pyodbc.connect('DSN=odbc_DSN_Configuration_Name',
autocommit=True)
print('Starting Connection')
curs = ch.cursor()
print('Successfuly Connected')
curs.execute(sql)
results = curs.fetchall()
#Getting Field
Header names
column_names = [i[0] for i in curs.description]
print(results)
f=open(file_name,"w")
c = csv.writer(f)
for database in results:
print(database.name)
c.writerow([database.name])
print("Connection
Closed")
f.close()
ch.close()
except Exception as error:
print ("An exception was thrown!")
print (str(error))
sql="select * from tableName"
file_name="C:\\Users\\ComputerName\\abc.csv"
connection(sql,file_name)
---------------------------------------------------------
Steps to define a web service in java using Eclipse
----------------------------------------------------------STEP1:-Got to File-->Dynamic Web Project name it Cal
Second STEP:-Then Right Click the project Cal and make a Class
Third step:-Then give java class java and package name in the following window in Eclipse
Fourth Step:-Then define the Class method which will be called as web service
Now Declare the Class Cal as web service Right the Class Cal to
New--> others --> select web service.
Then click next
Then define Client for the web service in following window automatically as shown below.
Press finishes Click next
Then Click next
Then Click next
Then next finish
Then call the method of the class (which web service) from the client as shown below.
#To get the Last Week First and Last day from Sunday to Saturdays dates in R Laguage.
if (weekdays(Sys.Date(),abbreviate=FALSE)=="Sunday"){
enddate=Sys.Date()-1
startdate=Sys.Date()-7
dayNum<-as.numeric(difftime(as.character(as.Date(Sys.Date(),tz = Sys.timezone())), "1970-01-01",units="days"))-7
print(startdate)
print(enddate)
print(dayNum)
}
if (weekdays(Sys.Date(),abbreviate=FALSE)=="Monday"){
enddate=Sys.Date()-2
startdate=Sys.Date()-8
dayNum<-as.numeric(difftime(as.character(as.Date(Sys.Date(),tz = Sys.timezone())), "1970-01-01",units="days"))-8
print(startdate)
print(enddate)
print(dayNum)
}
if (weekdays(Sys.Date(),abbreviate=FALSE)=="Tuesday"){
enddate=Sys.Date()-3
startdate=Sys.Date()-9
dayNum<-as.numeric(difftime(as.character(as.Date(Sys.Date(),tz = Sys.timezone())), "1970-01-01",units="days"))-9
print(startdate)
print(enddate)
print(dayNum)
}
if(weekdays(Sys.Date(),abbreviate=FALSE)=="Wednessday"){
enddate=Sys.Date()-4
startdate=Sys.Date()-10
dayNum<-as.numeric(difftime(as.character(as.Date(Sys.Date(),tz = Sys.timezone())), "1970-01-01",units="days"))-10
print(startdate)
print(enddate)
print(dayNum)
}
if (weekdays(Sys.Date(),abbreviate=FALSE)=="Thursday"){
enddate=Sys.Date()-5
startdate=Sys.Date()-11
dayNum<-as.numeric(difftime(as.character(as.Date(Sys.Date(),tz = Sys.timezone())), "1970-01-01",units="days"))-11
print(startdate)
print(enddate)
print(dayNum)
}
if (weekdays(Sys.Date(),abbreviate=FALSE)=="Friday"){
enddate=Sys.Date()-6
startdate=Sys.Date()-12
dayNum<-as.numeric(difftime(as.character(as.Date(Sys.Date(),tz = Sys.timezone())), "1970-01-01",units="days"))-12
print(startdate)
print(enddate)
print(dayNum)
}
if (weekdays(Sys.Date(),abbreviate=FALSE)=="Satureday"){
enddate=Sys.Date()-7
startdate=Sys.Date()-13
dayNum<-as.numeric(difftime(as.character(as.Date(Sys.Date(),tz = Sys.timezone())), "1970-01-01",units="days"))-13
print(startdate)
print(enddate)
print(dayNum)
}
