My Experiences with Databases & More

Fun with Python – Create Web Traffic using selenium & Python.

Posted by Sriram Sanka on October 4, 2022

In my Previous Post, I tried to Download the Content from the Blogs and store it in the File System, This Increased my Google Search Engine Stats and Blog traffic as well.

As you can See Max views are from Canada & India. With this, I thought of writing a Python Program to create traffic to my blog by reading my posts (so far ) using Selenium and Secure VPN.As I am connected to Canada VPN, you can see the Views below, Before and after .

In General QA Performs the same for App Testing Automation using selenium web driver and JS etc, Here I am using Python. Lets see the Code Part.

import codecs
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.common.keys import Keys
import time
import os
import pandas as pd
import requests
from lxml import etree
import random
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager

options = Options()
options.add_argument("start-maximized")
options.add_argument('--headless')
options.add_argument('--disable-gpu')
options.add_argument('--ignore-certificate-errors-spki-list')
options.add_argument('--ignore-certificate-errors')

options.add_argument("--incognito")
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()), options=options)

def create_traffic(url):
        website_random_URL = url
        driver.get(url)
        time.sleep(5)
        height = int(driver.execute_script("return document.documentElement.scrollHeight"))
        driver.execute_script('window.scrollBy(0,10)')
        time.sleep(10)
        

    
    

main_sitemap = 'https://ramoradba.com/sitemap.xml'
xmlDict = []
r = requests.get(main_sitemap)
root = etree.fromstring(r.content)
print ("The number of sitemap tags are {0}".format(len(root)))
for sitemap in root:
    children = sitemap.getchildren()
    xmlDict.append({'url': children[0].text})
    with open('links23.txt', 'a') as f:
        f.write( f'\n{children[0].text}')
        

pd.DataFrame(xmlDict)        
col_name = ['url']
df_url = pd.read_csv("links23.txt", names=col_name)
for row in df_url.url:
    print(row)
    create_traffic(row)

This Part is the Main Block, reading through my Blog post URL from the Links downloaded from the sitemap.

def create_traffic(url):
        website_random_URL = url
        driver.get(url)
        time.sleep(5)
        height = int(driver.execute_script("return document.documentElement.scrollHeight"))
        driver.execute_script('window.scrollBy(0,10)')
        time.sleep(10)

This Code Opens the URL in the Browser and scroll, The same can be configured using while loop forever by reading random posts from the Blog URL instead of all the Posts.

The More you execute the program, You will get more Traffic. Hope you like it.

Lets Connect to Ukraine, Kyiv and get the views from there.

Lets Execute and see the Progress…..

After Execution, Views from Ukraine have been Increased.

Follow me for more Interesting post in future , Twitter – TheRamOraDBA & linkedin-ramoradba

Posted in Python, Selenium, Web-Traffic, WebScraping | Tagged: Python, Scrapping, Selenium, Web-Traffic | Leave a Comment »

Hacking – FATDBA.COM ¯\_(ツ)_/¯ `ॐ`

Posted by Sriram Sanka on October 4, 2022

#Python #WebScraping

Just Kidding !!! Its not Hacking, this is known as WEB-SCRAPING using the Powerful Python.

What is web scraping?

Web scraping is the process of using bots to extract content and data from a website. Unlike screen scraping, which only copies pixels displayed onscreen, web scraping extracts underlying HTML code and, with it, data stored in a database.

You just need a Browser, Simple & a small Python Code to get the content from the Web. First Lets see the Parts of the Code and Verify.

Step 1 : Install & Load the Python Modules

import time
import os
import pandas as pd
import requests
from lxml import etree
import random
import urllib.request, urllib.error, urllib.parse
import urllib.parse
import sys
import urllib.request
import string

Step 2: Define function to get the Name of the Site/Blog to Make it as a Folder.

def get_host(url,delim):
    parsed_url = urllib.parse.urlparse(url)
    return(parsed_url.netloc.replace(delim, "_"))

Step 3: Define a Function to Get the Blog/Page Title

def findTitle(url,delim):
    webpage = urllib.request.urlopen(url).read()
    title = str(webpage).split('<title>')[1].split('</title>')[0]
    return title.replace(delim, "_").translate(str.maketrans('', '', string.punctuation))

Step 4: Define a Function to Generate a Unique string of a given length

def unq_str(len):
    N = len
    res = ''.join(random.choices(string.ascii_uppercase + string.digits, k=N))
    return(str(res))

Step 5: Write the Main Block to Download the Content from the Site/Blog

def Download_blog(path,url,enc):
    try:   
        response = urllib.request.urlopen(url)
        webContent = response.read().decode(enc)
        os.makedirs(path+'\\'+ str(get_host(url,".")), exist_ok=True)
        n=os.path.join(path+'\\'+ str(get_host(url,".")),findTitle(url," ") +'.html')
        f = open(n, 'w',encoding=enc)
        f.write(webContent)
        f.close
    except:
        n1=os.path.join(path+'\\'+  str(get_host(url,"."))+'Download_Error.log')
        f1 = open(n1, 'w',encoding=enc) 
        f1.write(url)
        f1.close

Step 6: Define Another Function to save the Blog posts into a file & Invoke the Main block to get the Blog Content.

def write_post_url_to_file(blog,path):        
    main_sitemap = blog+'/sitemap.xml'
    r = requests.get(main_sitemap)
    root = etree.fromstring(r.content)
    for sitemap in root:
        children = sitemap.getchildren()
        with open(str(path+'\\'+get_host(blog,".")) +'_blog_links.txt', 'a') as f:
            f.write( f'\n{children[0].text}')
    col_name = ['url']
    df_url = pd.read_csv(str(path+'\\'+get_host(blog,".")) +'_blog_links.txt', names=col_name)
    for row in df_url.url:
        print(row)
        Download_blog(path,row,'UTF-8')
        
write_post_url_to_file("https://fatdba.com","c:\\Users\\Dell\\Downloads\\blogs\\")

This will create a file with links and folder with blog name to store all the content/Posts Data.

Sample Output as follows

BOOM !!!

For more interesting posts you can follow me @ Twitter – TheRamOraDBA & linkedin-ramoradba

Posted in download_blogs, Linux, Python, WebScraping, Windows | Tagged: blog, HTML, Python, web-scraping | Leave a Comment »

Install MySQL 5.7 Community Server Using RPM Bundle- CENTOS 7

Posted by Sriram Sanka on October 2, 2022

To Install MySQL 5.7 we have To options.

Add the Yum Repository and Install using Yum
Download and Ship the Bundle/RPM and its dependencies to the Server and Install as RPM /Yum Local Install. i.e. either using yum Or rpm -ivh <RPM_NAME>

You can Download the RPM Or bundle Tar file from https://dev.mysql.com/downloads/repo/yum/

https://cdn.mysql.com/archives/mysql-5.7/mysql-5.7.9-1.el7.x86_64.rpm-bundle.tar – For 5.7

https://cdn.mysql.com//Downloads/MySQL-8.0/mysql-8.0.30-1.el7.x86_64.rpm-bundle.tar for 8.0

Few Useful links to Download your desired Version.

https://dev.mysql.com/downloads/repo/yum/

https://downloads.mysql.com/archives/community/

wget https://cdn.mysql.com/archives/mysql-5.7/mysql-5.7.9-1.el7.x86_64.rpm-bundle.tar

Install the Downloaded Packages/RPM Using Local-Install Option.

yum localinstall mysql-community-{server,client,common,libs}-* mysql-5.*

[root@localhost mysql_binaries]# yum localinstall mysql-community-{server,client,common,libs}-* mysql-5.*
Loaded plugins: fastestmirror, langpacks
Examining mysql-community-server-5.7.9-1.el7.x86_64.rpm: mysql-community-server-5.7.9-1.el7.x86_64
Marking mysql-community-server-5.7.9-1.el7.x86_64.rpm to be installed
Examining mysql-community-client-5.7.9-1.el7.x86_64.rpm: mysql-community-client-5.7.9-1.el7.x86_64
Marking mysql-community-client-5.7.9-1.el7.x86_64.rpm to be installed
Examining mysql-community-common-5.7.9-1.el7.x86_64.rpm: mysql-community-common-5.7.9-1.el7.x86_64
Marking mysql-community-common-5.7.9-1.el7.x86_64.rpm to be installed
Examining mysql-community-libs-5.7.9-1.el7.x86_64.rpm: mysql-community-libs-5.7.9-1.el7.x86_64
Marking mysql-community-libs-5.7.9-1.el7.x86_64.rpm to be installed
Examining mysql-community-libs-compat-5.7.9-1.el7.x86_64.rpm: mysql-community-libs-compat-5.7.9-1.el7.x86_64
Marking mysql-community-libs-compat-5.7.9-1.el7.x86_64.rpm to be installed
Skipping: mysql-5.*, filename does not end in .rpm.
Resolving Dependencies
--> Running transaction check
---> Package mariadb-libs.x86_64 1:5.5.68-1.el7 will be obsoleted
---> Package mysql-community-client.x86_64 0:5.7.9-1.el7 will be installed
---> Package mysql-community-common.x86_64 0:5.7.9-1.el7 will be installed
---> Package mysql-community-libs.x86_64 0:5.7.9-1.el7 will be obsoleting
---> Package mysql-community-libs-compat.x86_64 0:5.7.9-1.el7 will be obsoleting
---> Package mysql-community-server.x86_64 0:5.7.9-1.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

=============================================================================================================================================================================================================================================
 Package                                                      Arch                                    Version                                         Repository                                                                        Size
=============================================================================================================================================================================================================================================
Installing:
 mysql-community-client                                       x86_64                                  5.7.9-1.el7                                     /mysql-community-client-5.7.9-1.el7.x86_64                                       109 M
 mysql-community-common                                       x86_64                                  5.7.9-1.el7                                     /mysql-community-common-5.7.9-1.el7.x86_64                                       2.4 M
 mysql-community-libs                                         x86_64                                  5.7.9-1.el7                                     /mysql-community-libs-5.7.9-1.el7.x86_64                                         9.8 M
     replacing  mariadb-libs.x86_64 1:5.5.68-1.el7
 mysql-community-libs-compat                                  x86_64                                  5.7.9-1.el7                                     /mysql-community-libs-compat-5.7.9-1.el7.x86_64                                  9.2 M
     replacing  mariadb-libs.x86_64 1:5.5.68-1.el7
 mysql-community-server                                       x86_64                                  5.7.9-1.el7                                     /mysql-community-server-5.7.9-1.el7.x86_64                                       649 M

Transaction Summary
=============================================================================================================================================================================================================================================
Install  5 Packages

Total size: 779 M
Is this ok [y/d/N]: y
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : mysql-community-common-5.7.9-1.el7.x86_64                                                                                                                                                                                 1/6
  Installing : mysql-community-libs-5.7.9-1.el7.x86_64                                                                                                                                                                                   2/6
  Installing : mysql-community-client-5.7.9-1.el7.x86_64                                                                                                                                                                                 3/6
  Installing : mysql-community-server-5.7.9-1.el7.x86_64                                                                                                                                                                                 4/6
  Installing : mysql-community-libs-compat-5.7.9-1.el7.x86_64                                                                                                                                                                            5/6
  Erasing    : 1:mariadb-libs-5.5.68-1.el7.x86_64                                                                                                                                                                                        6/6
  Verifying  : mysql-community-client-5.7.9-1.el7.x86_64                                                                                                                                                                                 1/6
  Verifying  : mysql-community-libs-compat-5.7.9-1.el7.x86_64                                                                                                                                                                            2/6
  Verifying  : mysql-community-libs-5.7.9-1.el7.x86_64                                                                                                                                                                                   3/6
  Verifying  : mysql-community-server-5.7.9-1.el7.x86_64                                                                                                                                                                                 4/6
  Verifying  : mysql-community-common-5.7.9-1.el7.x86_64                                                                                                                                                                                 5/6
  Verifying  : 1:mariadb-libs-5.5.68-1.el7.x86_64                                                                                                                                                                                        6/6

Installed:
  mysql-community-client.x86_64 0:5.7.9-1.el7   mysql-community-common.x86_64 0:5.7.9-1.el7   mysql-community-libs.x86_64 0:5.7.9-1.el7   mysql-community-libs-compat.x86_64 0:5.7.9-1.el7   mysql-community-server.x86_64 0:5.7.9-1.el7

Replaced:
  mariadb-libs.x86_64 1:5.5.68-1.el7

Complete!

Enable MySQL Service using "service mysqld start"

Get the Temporary root password from /var/log/mysqld.log.

You can either change the Password using ALTER USER Or by Running Secure Installation option. i.e. mysql_secure_installation

[root@localhost mysql_binaries]# mysql_secure_installation

Securing the MySQL server deployment.

Enter password for user root:
Error: Access denied for user 'root'@'localhost' (using password: YES)
[root@localhost mysql_binaries]# mysql_secure_installation

Securing the MySQL server deployment.

Enter password for user root:

The existing password for the user account root has expired. Please set a new password.

New password:

Re-enter new password:
The 'validate_password' plugin is installed on the server.
The subsequent steps will run with the existing configuration
of the plugin.
Using existing password for root.

Estimated strength of the password: 100
Change the password for root ? ((Press y|Y for Yes, any other key for No) : n

 ... skipping.
By default, a MySQL installation has an anonymous user,
allowing anyone to log into MySQL without having to have
a user account created for them. This is intended only for
testing, and to make the installation go a bit smoother.
You should remove them before moving into a production
environment.

Remove anonymous users? (Press y|Y for Yes, any other key for No) : y
Success.


Normally, root should only be allowed to connect from
'localhost'. This ensures that someone cannot guess at
the root password from the network.

Disallow root login remotely? (Press y|Y for Yes, any other key for No) : y
Success.

By default, MySQL comes with a database named 'test' that
anyone can access. This is also intended only for testing,
and should be removed before moving into a production
environment.


Remove test database and access to it? (Press y|Y for Yes, any other key for No) : y
 - Dropping test database...
Success.

 - Removing privileges on test database...
Success.

Reloading the privilege tables will ensure that all changes
made so far will take effect immediately.

Reload privilege tables now? (Press y|Y for Yes, any other key for No) : y
Success.

All done!

You can Check the Installed Version By running the following

mysql> create database sriram;
Query OK, 1 row affected (0.00 sec)

mysql> use sriram;
Database changed
mysql> create table sriram_table (id int, name varchar(200));
Query OK, 0 rows affected (0.07 sec)

mysql> insert into sriram_table values(1,'Sriram');
Query OK, 1 row affected (0.02 sec)

mysql> insert into sriram_table values(2,'Sriram2');
Query OK, 1 row affected (0.03 sec)

mysql> insert into sriram_table values(3,'Sriram3');
Query OK, 1 row affected (0.02 sec)

mysql> commit;
Query OK, 0 rows affected (0.00 sec)

mysql> select * from sriram_table;
+------+---------+
| id   | name    |
+------+---------+
|    1 | Sriram  |
|    2 | Sriram2 |
|    3 | Sriram3 |
+------+---------+
3 rows in set (0.00 sec)

You can run mysql_upgrade to see the information on update is needed .

mysql_upgrade saves the MySQL version number in a file named mysql_upgrade_info in the data directory. This is used to quickly check whether all tables have been checked for this release so that table-checking can be skipped.

In Our Next Post we See How to Upgrade this to the Latest Using RPM Upgrade Or Using YUM .

References :

https://dev.mysql.com/doc/refman/5.7/en/installing.html

https://dev.mysql.com/doc/refman/8.0/en/installing.html

Hope this is useful, Happy Reading.

Posted in Installation, Linux, MySql | Tagged: centos, MySql, RPM, YUM | Leave a Comment »

Upgrade MySQL Server from 5.7 to 8 CentOS 7 Linux

Posted by Sriram Sanka on October 2, 2022

Quote from Mysql Website Upgrade Paths

Upgrade from MySQL 5.7 to 8.0 is supported. However, upgrade is only supported between General Availability (GA) releases. For MySQL 8.0, it is required that you upgrade from a MySQL 5.7 GA release (5.7.9 or higher). Upgrades from non-GA releases of MySQL 5.7 are not supported.
Upgrading to the latest release is recommended before upgrading to the next version. For example, upgrade to the latest MySQL 5.7 release before upgrading to MySQL 8.0.
Upgrade that skips versions is not supported. For example, upgrading directly from MySQL 5.6 to 8.0 is not supported.
Once a release series reaches General Availability (GA) status, upgrade within the release series (from one GA version to another GA version) is supported. For example, upgrading from MySQL 8.0.x to 8.0.y is supported. (Upgrade involving development-status non-GA releases is not supported.) Skipping a release is also supported. For example, upgrading from MySQL 8.0.x to 8.0.z is supported. MySQL 8.0.11 is the first GA status release within the MySQL 8.0 release series.

Reference: https://dev.mysql.com/doc/refman/8.0/en/upgrade-paths.html

Download the Required RPM and place them in the Server.

wget https://cdn.mysql.com//Downloads/MySQL-8.0/mysql-8.0.30-1.el7.x86_64.rpm-bundle.tar

Take the Backup of all the Databases and stop the MySQL Service.

Run the Following from Local Directory using yum localinstall mysql-community-{server,client,common,libs}-*

yum localinstall mysql-community-{server,client,common,libs}-*

[root@localhost mysql_binaries]# yum localinstall mysql-community-{server,client,common,libs}-*
Loaded plugins: fastestmirror, langpacks
Examining mysql-community-server-8.0.30-1.el7.x86_64.rpm: mysql-community-server-8.0.30-1.el7.x86_64
Marking mysql-community-server-8.0.30-1.el7.x86_64.rpm as an update to mysql-community-server-5.7.9-1.el7.x86_64
Examining mysql-community-server-debug-8.0.30-1.el7.x86_64.rpm: mysql-community-server-debug-8.0.30-1.el7.x86_64
Marking mysql-community-server-debug-8.0.30-1.el7.x86_64.rpm to be installed
Examining mysql-community-client-8.0.30-1.el7.x86_64.rpm: mysql-community-client-8.0.30-1.el7.x86_64
Marking mysql-community-client-8.0.30-1.el7.x86_64.rpm as an update to mysql-community-client-5.7.9-1.el7.x86_64
Examining mysql-community-client-plugins-8.0.30-1.el7.x86_64.rpm: mysql-community-client-plugins-8.0.30-1.el7.x86_64
Marking mysql-community-client-plugins-8.0.30-1.el7.x86_64.rpm to be installed
Examining mysql-community-common-8.0.30-1.el7.x86_64.rpm: mysql-community-common-8.0.30-1.el7.x86_64
Marking mysql-community-common-8.0.30-1.el7.x86_64.rpm as an update to mysql-community-common-5.7.9-1.el7.x86_64
Examining mysql-community-libs-8.0.30-1.el7.x86_64.rpm: mysql-community-libs-8.0.30-1.el7.x86_64
Marking mysql-community-libs-8.0.30-1.el7.x86_64.rpm as an update to mysql-community-libs-5.7.9-1.el7.x86_64
Examining mysql-community-libs-compat-8.0.30-1.el7.x86_64.rpm: mysql-community-libs-compat-8.0.30-1.el7.x86_64
Marking mysql-community-libs-compat-8.0.30-1.el7.x86_64.rpm as an update to mysql-community-libs-compat-5.7.9-1.el7.x86_64
Resolving Dependencies
--> Running transaction check
---> Package mysql-community-client.x86_64 0:5.7.9-1.el7 will be updated
---> Package mysql-community-client.x86_64 0:8.0.30-1.el7 will be an update
---> Package mysql-community-client-plugins.x86_64 0:8.0.30-1.el7 will be installed
---> Package mysql-community-common.x86_64 0:5.7.9-1.el7 will be updated
---> Package mysql-community-common.x86_64 0:8.0.30-1.el7 will be an update
---> Package mysql-community-libs.x86_64 0:5.7.9-1.el7 will be updated
---> Package mysql-community-libs.x86_64 0:8.0.30-1.el7 will be an update
---> Package mysql-community-libs-compat.x86_64 0:5.7.9-1.el7 will be updated
---> Package mysql-community-libs-compat.x86_64 0:8.0.30-1.el7 will be an update
---> Package mysql-community-server.x86_64 0:5.7.9-1.el7 will be updated
---> Package mysql-community-server.x86_64 0:8.0.30-1.el7 will be an update
---> Package mysql-community-server-debug.x86_64 0:8.0.30-1.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

=============================================================================================================================================================================================================================================
 Package                                                       Arch                                  Version                                        Repository                                                                          Size
=============================================================================================================================================================================================================================================
Installing:
 mysql-community-client-plugins                                x86_64                                8.0.30-1.el7                                   /mysql-community-client-plugins-8.0.30-1.el7.x86_64                                 14 M
 mysql-community-server-debug                                  x86_64                                8.0.30-1.el7                                   /mysql-community-server-debug-8.0.30-1.el7.x86_64                                  115 M
Updating:
 mysql-community-client                                        x86_64                                8.0.30-1.el7                                   /mysql-community-client-8.0.30-1.el7.x86_64                                         71 M
 mysql-community-common                                        x86_64                                8.0.30-1.el7                                   /mysql-community-common-8.0.30-1.el7.x86_64                                        9.9 M
 mysql-community-libs                                          x86_64                                8.0.30-1.el7                                   /mysql-community-libs-8.0.30-1.el7.x86_64                                          7.6 M
 mysql-community-libs-compat                                   x86_64                                8.0.30-1.el7                                   /mysql-community-libs-compat-8.0.30-1.el7.x86_64                                   3.7 M
 mysql-community-server                                        x86_64                                8.0.30-1.el7                                   /mysql-community-server-8.0.30-1.el7.x86_64                                        241 M

Transaction Summary
=============================================================================================================================================================================================================================================
Install  2 Packages
Upgrade  5 Packages

Total size: 462 M
Is this ok [y/d/N]: y
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : mysql-community-client-plugins-8.0.30-1.el7.x86_64                                                                                                                                                                       1/12
  Updating   : mysql-community-common-8.0.30-1.el7.x86_64                                                                                                                                                                               2/12
  Updating   : mysql-community-libs-8.0.30-1.el7.x86_64                                                                                                                                                                                 3/12
  Updating   : mysql-community-client-8.0.30-1.el7.x86_64                                                                                                                                                                               4/12
  Updating   : mysql-community-server-8.0.30-1.el7.x86_64                                                                                                                                                                               5/12
  Installing : mysql-community-server-debug-8.0.30-1.el7.x86_64                                                                                                                                                                         6/12
  Updating   : mysql-community-libs-compat-8.0.30-1.el7.x86_64                                                                                                                                                                          7/12
  Cleanup    : mysql-community-server-5.7.9-1.el7.x86_64                                                                                                                                                                                8/12
  Cleanup    : mysql-community-client-5.7.9-1.el7.x86_64                                                                                                                                                                                9/12
  Cleanup    : mysql-community-libs-compat-5.7.9-1.el7.x86_64                                                                                                                                                                          10/12
  Cleanup    : mysql-community-libs-5.7.9-1.el7.x86_64                                                                                                                                                                                 11/12
  Cleanup    : mysql-community-common-5.7.9-1.el7.x86_64                                                                                                                                                                               12/12
  Verifying  : mysql-community-common-8.0.30-1.el7.x86_64                                                                                                                                                                               1/12
  Verifying  : mysql-community-client-plugins-8.0.30-1.el7.x86_64                                                                                                                                                                       2/12
  Verifying  : mysql-community-libs-8.0.30-1.el7.x86_64                                                                                                                                                                                 3/12
  Verifying  : mysql-community-client-8.0.30-1.el7.x86_64                                                                                                                                                                               4/12
  Verifying  : mysql-community-libs-compat-8.0.30-1.el7.x86_64                                                                                                                                                                          5/12
  Verifying  : mysql-community-server-8.0.30-1.el7.x86_64                                                                                                                                                                               6/12
  Verifying  : mysql-community-server-debug-8.0.30-1.el7.x86_64                                                                                                                                                                         7/12
  Verifying  : mysql-community-client-5.7.9-1.el7.x86_64                                                                                                                                                                                8/12
  Verifying  : mysql-community-libs-5.7.9-1.el7.x86_64                                                                                                                                                                                  9/12
  Verifying  : mysql-community-server-5.7.9-1.el7.x86_64                                                                                                                                                                               10/12
  Verifying  : mysql-community-common-5.7.9-1.el7.x86_64                                                                                                                                                                               11/12
  Verifying  : mysql-community-libs-compat-5.7.9-1.el7.x86_64                                                                                                                                                                          12/12

Installed:
  mysql-community-client-plugins.x86_64 0:8.0.30-1.el7                                                                   mysql-community-server-debug.x86_64 0:8.0.30-1.el7

Updated:
  mysql-community-client.x86_64 0:8.0.30-1.el7  mysql-community-common.x86_64 0:8.0.30-1.el7  mysql-community-libs.x86_64 0:8.0.30-1.el7  mysql-community-libs-compat.x86_64 0:8.0.30-1.el7  mysql-community-server.x86_64 0:8.0.30-1.el7

Complete!

This completes the Upgrade from 5.7 to 8.0(Or Latest) Start the Service again and verify.

[root@localhost mysql_binaries]# service mysqld start
Redirecting to /bin/systemctl start mysqld.service
[root@localhost mysql_binaries]# service mysqld status
Redirecting to /bin/systemctl status mysqld.service
● mysqld.service - MySQL Server
   Loaded: loaded (/usr/lib/systemd/system/mysqld.service; enabled; vendor preset: disabled)
   Active: active (running) since Sun 2022-10-02 03:54:08 EDT; 4s ago
     Docs: man:mysqld(8)
           http://dev.mysql.com/doc/refman/en/using-systemd.html
  Process: 21468 ExecStartPre=/usr/bin/mysqld_pre_systemd (code=exited, status=0/SUCCESS)
 Main PID: 21502 (mysqld)
   Status: "Server is operational"
    Tasks: 39
   CGroup: /system.slice/mysqld.service
           └─21502 /usr/sbin/mysqld

Oct 02 03:53:16 localhost.localdomain systemd[1]: Starting MySQL Server...
Oct 02 03:54:08 localhost.localdomain systemd[1]: Started MySQL Server.

Note You can also use rpm -Uvh *.rpm to upgrade the Version.

Connect to the Database and verify if there are any errors.

[root@localhost mysql_binaries]# mysql -uroot -p************
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 11
Server version: 8.0.30 MySQL Community Server - GPL

Copyright (c) 2000, 2022, Oracle and/or its affiliates.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| sriram             |
| sys                |
+--------------------+
5 rows in set (0.00 sec)

mysql> use sriram;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> show tables;
+------------------+
| Tables_in_sriram |
+------------------+
| sriram_table     |
+------------------+
1 row in set (0.01 sec)

mysql> select * from sriram_table;
+------+---------+
| id   | name    |
+------+---------+
|    1 | Sriram  |
|    2 | Sriram2 |
|    3 | Sriram3 |
+------+---------+
3 rows in set (0.00 sec)

mysql> show variables like '%version%';
+--------------------------+------------------------------+
| Variable_name            | Value                        |
+--------------------------+------------------------------+
| admin_tls_version        | TLSv1.2                      |
| immediate_server_version | 999999                       |
| innodb_version           | 8.0.30                       |
| original_server_version  | 999999                       |
| protocol_version         | 10                           |
| replica_type_conversions |                              |
| slave_type_conversions   |                              |
| tls_version              | TLSv1.2                      |
| version                  | 8.0.30                       |
| version_comment          | MySQL Community Server - GPL |
| version_compile_machine  | x86_64                       |
| version_compile_os       | Linux                        |
| version_compile_zlib     | 1.2.12                       |
+--------------------------+------------------------------+
13 rows in set (0.00 sec)

mysql> SELECT VERSION();
+-----------+
| VERSION() |
+-----------+
| 8.0.30    |
+-----------+
1 row in set (0.00 sec)

mysql>

Since Mysql_upgrade is deprecated and unneeded in Version 8.0.16 and later, you can run mysql_check if needed.

[root@localhost mysql_binaries]# mysql_upgrade -uroot -p
The mysql_upgrade client is now deprecated. The actions executed by the upgrade client are now done by the server.
To upgrade, please start the new MySQL binary with the older data directory. Repairing user tables is done automatically. Restart is not required after upgrade.
The upgrade process automatically starts on running a new MySQL binary with an older data directory. To avoid accidental upgrades, please use the --upgrade=NONE option with the MySQL binary. The option --upgrade=FORCE is also provided to run the server upgrade sequence on demand.
It may be possible that the server upgrade fails due to a number of reasons. In that case, the upgrade sequence will run again during the next MySQL server start. If the server upgrade fails repeatedly, the server can be started with the --upgrade=MINIMAL option to start the server without executing the upgrade sequence, thus allowing users to manually rectify the problem.

Option 2: Follow the Below steps to Complete the Upgrade Steps

Take a backup of all the Databases & Parameter /Variable Values assigned.

Stop the Service.

Remove the Installed Old Version RPM by running rpm -qa mysql* and rpm -e <RPM_LIST>

Install the Latest Version

Restore the Backup from above

Add and Apply variable as before as supported

Conclusion : We have successfully completed the MySQL Community Server Version Upgrade on a standalone Server from 5.7 to 8.0.30

Posted in Installation, Linux, MySql, Upgrade, Upgrade 5.7 to 8 | Tagged: centos 7, Linux, MySql, RPM, Upgrade, YUM | Leave a Comment »

Docker Part 1 – Install Docker CE and Docker Compose in Oracle Linux Server 8.6 – OCI-IAAS

Posted by Sriram Sanka on September 29, 2022

In this Post, lets see how to Install Docker CE . I am using Oracle Linux Server 8.6 (Free Instance Offered by Oracle Cloud) .

Docker lets you build, test, and deploy applications quickly, Docker is a software platform that allows you to build, test, and deploy applications quickly. Docker packages software into standardized units called containers that have everything the software needs to run including libraries, system tools, code, and runtime. Using Docker, you can quickly deploy and scale applications into any environment and know your code will run.

Docker Compose is a tool that was developed to help define and share multi-container applications. With Compose, we can create a YAML file to define the services and with a single command, can spin everything up or tear it all down.

The big advantage of using Compose is you can define your application stack in a file, keep it at the root of your project repo (it’s now version controlled), and easily enable someone else to contribute to your project. Someone would only need to clone your repo and start the compose app

Note : If there is no repo available in your OS, try adding EPEL repo and Try again.

[root@certbot conf.d]# dnf config-manager --add-repo=https://download.docker.com/linux/centos/docker-ce.repo
Adding repo from: https://download.docker.com/linux/centos/docker-ce.repo

Once the Repo is Added, we are good to go …..

run this command “dnf install docker-ce docker-ce-cli containerd.io”

[root@certbot conf.d]# dnf install docker-ce docker-ce-cli containerd.io
Last metadata expiration check: 0:00:57 ago on Wed 21 Sep 2022 08:05:23 PM GMT.
Dependencies resolved.
=============================================================================================================================================================================================================================================
 Package                                                     Architecture                             Version                                                                      Repository                                           Size
=============================================================================================================================================================================================================================================
Installing:
 containerd.io                                               x86_64                                   1.6.8-3.1.el8                                                                docker-ce-stable                                     33 M
 docker-ce                                                   x86_64                                   3:20.10.18-3.el8                                                             docker-ce-stable                                     21 M
 docker-ce-cli                                               x86_64                                   1:20.10.18-3.el8                                                             docker-ce-stable                                     30 M
Installing dependencies:
 container-selinux                                           noarch                                   2:2.188.0-1.module+el8.6.0+20721+d8d917a9                                    ol8_appstream                                        59 k
 docker-ce-rootless-extras                                   x86_64                                   20.10.18-3.el8                                                               docker-ce-stable                                    4.6 M
 fuse-common                                                 x86_64                                   3.3.0-15.0.2.el8                                                             ol8_baseos_latest                                    22 k
 fuse-overlayfs                                              x86_64                                   1.9-1.module+el8.6.0+20721+d8d917a9                                          ol8_appstream                                        73 k
 fuse3                                                       x86_64                                   3.3.0-15.0.2.el8                                                             ol8_baseos_latest                                    55 k
 fuse3-libs                                                  x86_64                                   3.3.0-15.0.2.el8                                                             ol8_baseos_latest                                    95 k
 libcgroup                                                   x86_64                                   0.41-19.el8                                                                  ol8_baseos_latest                                    70 k
 libslirp                                                    x86_64                                   4.4.0-1.module+el8.6.0+20721+d8d917a9                                        ol8_appstream                                        70 k
 slirp4netns                                                 x86_64                                   1.2.0-2.module+el8.6.0+20721+d8d917a9                                        ol8_appstream                                        54 k
Installing weak dependencies:
 docker-scan-plugin                                          x86_64                                   0.17.0-3.el8                                                                 docker-ce-stable                                    3.8 M
Enabling module streams:
 container-tools                                                                                      ol8

Transaction Summary
=============================================================================================================================================================================================================================================
Install  13 Packages

Total download size: 92 M
Installed size: 365 M
Is this ok [y/N]: y
Downloading Packages:
(1/13): docker-ce-20.10.18-3.el8.x86_64.rpm                                                                                                                                                                   12 MB/s |  21 MB     00:01
(2/13): docker-ce-rootless-extras-20.10.18-3.el8.x86_64.rpm                                                                                                                                                  8.1 MB/s | 4.6 MB     00:00
(3/13): docker-scan-plugin-0.17.0-3.el8.x86_64.rpm                                                                                                                                                           9.5 MB/s | 3.8 MB     00:00
(4/13): docker-ce-cli-20.10.18-3.el8.x86_64.rpm                                                                                                                                                               10 MB/s |  30 MB     00:02
(5/13): fuse-common-3.3.0-15.0.2.el8.x86_64.rpm                                                                                                                                                               78 kB/s |  22 kB     00:00
(6/13): fuse3-3.3.0-15.0.2.el8.x86_64.rpm                                                                                                                                                                    650 kB/s |  55 kB     00:00
(7/13): fuse3-libs-3.3.0-15.0.2.el8.x86_64.rpm                                                                                                                                                               7.8 MB/s |  95 kB     00:00
(8/13): container-selinux-2.188.0-1.module+el8.6.0+20721+d8d917a9.noarch.rpm                                                                                                                                  11 MB/s |  59 kB     00:00
(9/13): fuse-overlayfs-1.9-1.module+el8.6.0+20721+d8d917a9.x86_64.rpm                                                                                                                                         10 MB/s |  73 kB     00:00
(10/13): containerd.io-1.6.8-3.1.el8.x86_64.rpm                                                                                                                                                              9.9 MB/s |  33 MB     00:03
(11/13): libcgroup-0.41-19.el8.x86_64.rpm                                                                                                                                                                    206 kB/s |  70 kB     00:00
(12/13): libslirp-4.4.0-1.module+el8.6.0+20721+d8d917a9.x86_64.rpm                                                                                                                                           219 kB/s |  70 kB     00:00
(13/13): slirp4netns-1.2.0-2.module+el8.6.0+20721+d8d917a9.x86_64.rpm                                                                                                                                        2.2 MB/s |  54 kB     00:00
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                                                                         27 MB/s |  92 MB     00:03
Docker CE Stable - x86_64                                                                                                                                                                                     85 kB/s | 1.6 kB     00:00
Importing GPG key 0x621E9F35:
 Userid     : "Docker Release (CE rpm) <docker@docker.com>"
 Fingerprint: 060A 61C5 1B55 8A7F 742B 77AA C52F EB6B 621E 9F35
 From       : https://download.docker.com/linux/centos/gpg
Is this ok [y/N]: Y
Key imported successfully
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                                                                                                                                                                                     1/1
  Installing       : docker-scan-plugin-0.17.0-3.el8.x86_64                                                                                                                                                                             1/13
  Running scriptlet: docker-scan-plugin-0.17.0-3.el8.x86_64                                                                                                                                                                             1/13
  Installing       : docker-ce-cli-1:20.10.18-3.el8.x86_64                                                                                                                                                                              2/13
  Running scriptlet: docker-ce-cli-1:20.10.18-3.el8.x86_64                                                                                                                                                                              2/13
  Running scriptlet: container-selinux-2:2.188.0-1.module+el8.6.0+20721+d8d917a9.noarch                                                                                                                                                 3/13
  Installing       : container-selinux-2:2.188.0-1.module+el8.6.0+20721+d8d917a9.noarch                                                                                                                                                 3/13
  Running scriptlet: container-selinux-2:2.188.0-1.module+el8.6.0+20721+d8d917a9.noarch                                                                                                                                                 3/13
  Installing       : fuse3-libs-3.3.0-15.0.2.el8.x86_64                                                                                                                                                                                 4/13
  Running scriptlet: fuse3-libs-3.3.0-15.0.2.el8.x86_64                                                                                                                                                                                 4/13
  Installing       : containerd.io-1.6.8-3.1.el8.x86_64                                                                                                                                                                                 5/13
  Running scriptlet: containerd.io-1.6.8-3.1.el8.x86_64                                                                                                                                                                                 5/13
  Installing       : libslirp-4.4.0-1.module+el8.6.0+20721+d8d917a9.x86_64                                                                                                                                                              6/13
  Installing       : slirp4netns-1.2.0-2.module+el8.6.0+20721+d8d917a9.x86_64                                                                                                                                                           7/13
  Running scriptlet: libcgroup-0.41-19.el8.x86_64                                                                                                                                                                                       8/13
  Installing       : libcgroup-0.41-19.el8.x86_64                                                                                                                                                                                       8/13
  Running scriptlet: libcgroup-0.41-19.el8.x86_64                                                                                                                                                                                       8/13
  Installing       : fuse-common-3.3.0-15.0.2.el8.x86_64                                                                                                                                                                                9/13
  Installing       : fuse3-3.3.0-15.0.2.el8.x86_64                                                                                                                                                                                     10/13
  Installing       : fuse-overlayfs-1.9-1.module+el8.6.0+20721+d8d917a9.x86_64                                                                                                                                                         11/13
  Running scriptlet: fuse-overlayfs-1.9-1.module+el8.6.0+20721+d8d917a9.x86_64                                                                                                                                                         11/13
  Installing       : docker-ce-rootless-extras-20.10.18-3.el8.x86_64                                                                                                                                                                   12/13
  Running scriptlet: docker-ce-rootless-extras-20.10.18-3.el8.x86_64                                                                                                                                                                   12/13
  Installing       : docker-ce-3:20.10.18-3.el8.x86_64                                                                                                                                                                                 13/13
  Running scriptlet: docker-ce-3:20.10.18-3.el8.x86_64                                                                                                                                                                                 13/13
  Running scriptlet: container-selinux-2:2.188.0-1.module+el8.6.0+20721+d8d917a9.noarch                                                                                                                                                13/13
  Running scriptlet: docker-ce-3:20.10.18-3.el8.x86_64                                                                                                                                                                                 13/13
  Verifying        : containerd.io-1.6.8-3.1.el8.x86_64                                                                                                                                                                                 1/13
  Verifying        : docker-ce-3:20.10.18-3.el8.x86_64                                                                                                                                                                                  2/13
  Verifying        : docker-ce-cli-1:20.10.18-3.el8.x86_64                                                                                                                                                                              3/13
  Verifying        : docker-ce-rootless-extras-20.10.18-3.el8.x86_64                                                                                                                                                                    4/13
  Verifying        : docker-scan-plugin-0.17.0-3.el8.x86_64                                                                                                                                                                             5/13
  Verifying        : fuse-common-3.3.0-15.0.2.el8.x86_64                                                                                                                                                                                6/13
  Verifying        : fuse3-3.3.0-15.0.2.el8.x86_64                                                                                                                                                                                      7/13
  Verifying        : fuse3-libs-3.3.0-15.0.2.el8.x86_64                                                                                                                                                                                 8/13
  Verifying        : libcgroup-0.41-19.el8.x86_64                                                                                                                                                                                       9/13
  Verifying        : container-selinux-2:2.188.0-1.module+el8.6.0+20721+d8d917a9.noarch                                                                                                                                                10/13
  Verifying        : fuse-overlayfs-1.9-1.module+el8.6.0+20721+d8d917a9.x86_64                                                                                                                                                         11/13
  Verifying        : libslirp-4.4.0-1.module+el8.6.0+20721+d8d917a9.x86_64                                                                                                                                                             12/13
  Verifying        : slirp4netns-1.2.0-2.module+el8.6.0+20721+d8d917a9.x86_64                                                                                                                                                          13/13

Installed:
  container-selinux-2:2.188.0-1.module+el8.6.0+20721+d8d917a9.noarch          containerd.io-1.6.8-3.1.el8.x86_64              docker-ce-3:20.10.18-3.el8.x86_64            docker-ce-cli-1:20.10.18-3.el8.x86_64
  docker-ce-rootless-extras-20.10.18-3.el8.x86_64                             docker-scan-plugin-0.17.0-3.el8.x86_64          fuse-common-3.3.0-15.0.2.el8.x86_64          fuse-overlayfs-1.9-1.module+el8.6.0+20721+d8d917a9.x86_64
  fuse3-3.3.0-15.0.2.el8.x86_64                                               fuse3-libs-3.3.0-15.0.2.el8.x86_64              libcgroup-0.41-19.el8.x86_64                 libslirp-4.4.0-1.module+el8.6.0+20721+d8d917a9.x86_64
  slirp4netns-1.2.0-2.module+el8.6.0+20721+d8d917a9.x86_64

Complete!

Enable and Start the docker Service

[root@certbot conf.d]# systemctl enable docker
Created symlink /etc/systemd/system/multi-user.target.wants/docker.service → /usr/lib/systemd/system/docker.service.
[root@certbot conf.d]# systemctl start docker

Command “Docker PS” shows the running dockers.

[root@certbot conf.d]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

Lets Install Docker Compose as follows

[root@certbot conf.d]# curl -L "https://github.com/docker/compose/releases/download/1.23.2/docker-compose-$(uname -s)-$(uname -m)" -o docker-compose
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 11.2M  100 11.2M    0     0  44.9M      0 --:--:-- --:--:-- --:--:-- 44.9M
[root@certbot conf.d]# mv docker-compose /usr/local/bin && sudo chmod +x /usr/local/bin/docker-compose
[root@certbot conf.d]# docker-compose version
docker-compose version 1.23.2, build 1110ad01
docker-py version: 3.6.0
CPython version: 3.6.7
OpenSSL version: OpenSSL 1.1.0f  25 May 2017
[root@certbot conf.d]#

Add user to docker group (if not already added)
sudo usermod -aG docker $USER
2. create a symbolic link to /usr/bin using the following command
$ sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
3. Restart docker service
$ sudo service docker restart

Lets see the Available Commands with Docker help, We can discuss more on these commands in the Next Part

See 'docker --help'.

Usage:  docker [OPTIONS] COMMAND

A self-sufficient runtime for containers

Options:
      --config string      Location of client config files (default "/root/.docker")
  -c, --context string     Name of the context to use to connect to the daemon (overrides DOCKER_HOST env var and default context set with "docker context use")
  -D, --debug              Enable debug mode
  -H, --host list          Daemon socket(s) to connect to
  -l, --log-level string   Set the logging level ("debug"|"info"|"warn"|"error"|"fatal") (default "info")
      --tls                Use TLS; implied by --tlsverify
      --tlscacert string   Trust certs signed only by this CA (default "/root/.docker/ca.pem")
      --tlscert string     Path to TLS certificate file (default "/root/.docker/cert.pem")
      --tlskey string      Path to TLS key file (default "/root/.docker/key.pem")
      --tlsverify          Use TLS and verify the remote
  -v, --version            Print version information and quit

Management Commands:
  app*        Docker App (Docker Inc., v0.9.1-beta3)
  builder     Manage builds
  buildx*     Docker Buildx (Docker Inc., v0.7.1-docker)
  config      Manage Docker configs
  container   Manage containers
  context     Manage contexts
  image       Manage images
  manifest    Manage Docker image manifests and manifest lists
  network     Manage networks
  node        Manage Swarm nodes
  plugin      Manage plugins
  scan*       Docker Scan (Docker Inc., v0.12.0)
  secret      Manage Docker secrets
  service     Manage services
  stack       Manage Docker stacks
  swarm       Manage Swarm
  system      Manage Docker
  trust       Manage trust on Docker images
  volume      Manage volumes

Commands:
  attach      Attach local standard input, output, and error streams to a running container
  build       Build an image from a Dockerfile
  commit      Create a new image from a container's changes
  cp          Copy files/folders between a container and the local filesystem
  create      Create a new container
  diff        Inspect changes to files or directories on a container's filesystem
  events      Get real time events from the server
  exec        Run a command in a running container
  export      Export a container's filesystem as a tar archive
  history     Show the history of an image
  images      List images
  import      Import the contents from a tarball to create a filesystem image
  info        Display system-wide information
  inspect     Return low-level information on Docker objects
  kill        Kill one or more running containers
  load        Load an image from a tar archive or STDIN
  login       Log in to a Docker registry
  logout      Log out from a Docker registry
  logs        Fetch the logs of a container
  pause       Pause all processes within one or more containers
  port        List port mappings or a specific mapping for the container
  ps          List containers
  pull        Pull an image or a repository from a registry
  push        Push an image or a repository to a registry
  rename      Rename a container
  restart     Restart one or more containers
  rm          Remove one or more containers
  rmi         Remove one or more images
  run         Run a command in a new container
  save        Save one or more images to a tar archive (streamed to STDOUT by default)
  search      Search the Docker Hub for images
  start       Start one or more stopped containers
  stats       Display a live stream of container(s) resource usage statistics
  stop        Stop one or more running containers
  tag         Create a tag TARGET_IMAGE that refers to SOURCE_IMAGE
  top         Display the running processes of a container
  unpause     Unpause all processes within one or more containers
  update      Update configuration of one or more containers
  version     Show the Docker version information
  wait        Block until one or more containers stop, then print their exit codes

Run 'docker COMMAND --help' for more information on a command.

To get more help with docker, check out our guides at https://docs.docker.com/go/guides/

Posted in DevOps, Docker, Docker CE, Docker Compose, Linux | Tagged: DevOps, Docker, Docker-CE, Docker-Compose | Leave a Comment »

Install & Configure SSL for Apache-Nginx using Lets Encrypt-CertBot.

Posted by Sriram Sanka on September 28, 2022

One can Install Apache and Nginx using YUM Or DNF in the Selected Unix flavor whereas by default it is a non-secure sub-domain when you access.

Lets encrypt offers free SSL which can be configured to get the SSL for your domain irrespective of Private Or Public Domains. In case of Private Domains , you just need to add a text Entry to Pass the Validations.

For this I am using Oracle Cloud Instance(Always Free).

Change the Host Name using hostnamectl as below

[root@certbot ~]# hostnamectl set-hostname certbot.ramoradba.com
[root@certbot ~]# hostname

As its the Initial Login after Instance provision , run the yum update and Install Apache and/or Nginx as per your choice.

Run Yum Update and make sure everything updated without any issues.

Install Apache-httpd using yum repo.

Try Access the IP/Hostname to see the Installed Apache Default Page.

Install Nginx using Yum

Try to Access Nginx from the Browser

Enable EPEL Repo to Configure Snap and Certbot

Enable the Socket and run the below to Install certbot ,

systemctl enable --now snapd.socket

ln -s /var/lib/snapd/snap /snap

snap install certbot --classic

Restart your session to get the Certbot ,You can Configure SSL for Nginx Or Apache as below, Add an Entry in you domain controller for your IP matching with the Host Name Configured.

You can Either Configure SSL and Install Or Choose the certonly Option to Get the Certificates only, you can configure your SSL.conf as per your webserver configuration
Also In case , your System is not internet facing, You can choose the Preferred Challenges as either http or DNS You can review the Supported Challenged here https://letsencrypt.org/docs/challenge-types/

certbot --nginx -d <subdomain>.ramoradba.com
certbot --apache -d <subdomain>.ramoradba.com

Make Sure you have a Virtual_Host entry available with the domain you chose., Otherwise it will fail.

<VirtualHost *:80>
    ServerName certbot.ramoradba.com
    ServerAdmin webmaster@localhost
    DocumentRoot /var/www/html

    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

certbot --apache -d certbot.ramoradba.com

Reload your browser session to see the Installed Certificate .

Thanks for your visit, Hope you like it.

Posted in Apache, Installation, Linux, Nginx, Security, ssl | Tagged: Apache, certbot, letsencrypt, Linux, Nginx, OCI, ssl | Leave a Comment »

Python Basics – Part 1

Posted by Sriram Sanka on September 17, 2022

Language Introduction

Python is a dynamic, interpreted (bytecode-compiled) language. There are no type declarations of variables, parameters, functions, or methods in source code. This makes the code short and flexible, and you lose the compile-time type checking of the source code. Python tracks the types of all values at runtime and flags code that does not make sense as it runs.

https://www.edureka.co/blog/introduction-to-python/

In the Below Sections I have attached couple of reference Documents and Practice Notes for your reference. To obtain the contents, Rename the file Extension from txt to “ipynb” , which can be accessed using Jupyter Or Anaconda etc.

installingpython.1 Download

String Split

Description

Split the string input_str = ‘Kumar_Ravi_003’ to the person’s second name, first name and unique customer code. In this example, second_name= ‘Kumar’, first_name= ‘Ravi’, customer_code = ‘003’.

input_str = input('data')
first_name = input_str[6:10]
second_name = input_str[0:5]
customer_code = input_str[-3:]
print(first_name)
print(second_name)
print(customer_code)

string -lstrip()

input_str = input('Enter Input : ')
final_str = input_str.lstrip()
print(final_str)

introduction-datatypes Download

List is a collection which is ordered and changeable. Allows duplicate members.

Tuple is a collection which is ordered and unchangeable. Allows duplicate members.

Set is a collection which is unordered, unchangeable*, and unindexed. No duplicate members.

Dictionary is a collection which is ordered** and changeable. No duplicate members.

List to String

Description

Convert a list [‘Pythons syntax is easy to learn’, ‘Pythons syntax is very clear’] to a string using ‘&’. The sample output of this string will be:

Pythons syntax is easy to learn & Pythons syntax is very clear

Note that there is a space on both sides of ‘&’ (as usual in English sentences).

l =[]
l.append('Pythons syntax is easy to learn')
l.append(' Pythons syntax is very clear')
print('This is the List ',l)
input_str = l
string_1 = " & ".join(input_str)
print('This is Combined String ',string_1)

listcomprehensionsinpython Download

lists Download

tuples Download

dictionaries Download

sets Download

reference-workbook-2-data-structures Download

conditional-ifelse Download

looping-for-while Download

comprehensions Download

functions Download

map-reduce-filter Download

References

https://python-course.eu/advanced-python/lambda-filter-reduce-map.php

https://book.pythontips.com/en/latest/map_filter.html

https://python.swaroopch.com/functions.html

https://anh.cs.luc.edu/python/hands-on/3.1/handsonHtml/functions.html

https://treyhunner.com/2015/12/python-list-comprehensions-now-in-color/

https://python-3-patterns-idioms-test.readthedocs.io/en/latest/Comprehensions.html

https://docs.python.org/3/tutorial/controlflow.html

https://docs.python.org/3/reference/compound_stmts.html

https://docs.python.org/3/tutorial/datastructures.html

https://jupyter-notebook-beginner-guide.readthedocs.io/en/latest/

https://jupyter-notebook-beginner-guide.readthedocs.io/en/latest/what_is_jupyter.html

https://python.swaroopch.com/

https://docs.python-guide.org/intro/learning/

https://www.simplilearn.com/tutorials/python-tutorial

https://developers.google.com/edu/python/lists

https://developers.google.com/edu/python/introduction

Posted in Anaconda, Python | Tagged: Anaconda, Basics, Python | Leave a Comment »

SQL Formatter – Using Java/PL/SQL by Sayan Malakshinov

Posted by Sriram Sanka on September 17, 2022

It is very easy to Format SQL using Third Party tools, Recently I found one source to get the SQL Formatted from the Sql prompt without connecting to any Third Party GUI Tools.

I prefer to use SQLPLUS more than GUI tools., You can see the code formatted beautifully and very readable.

Step 1: Load oracle.dbtools-common.jar into the Database.

Microsoft Windows [Version 10.0.19044.2006]
(c) Microsoft Corporation. All rights reserved.

C:\Users\Dell>loadjava -u sriram/zaqmlp123 C:\app\Dell\product\21c\dbhomeXE\jlib\oracle.dbtools-common.jar

Step 2: Grant required java Privileges to the User and Create Java and Write a wrapper Package using the java

C:\Users\Dell>sqlplus /nolog

SQL*Plus: Release 21.0.0.0.0 - Production on Sat Sep 17 16:52:53 2022
Version 21.3.0.0.0

Copyright (c) 1982, 2021, Oracle.  All rights reserved.

SQL> conn sriram@xepdb1
Enter password:
Connected.
SQL>
SQL> exec dbms_java.grant_permission( 'SRIRAM', 'SYS:java.lang.RuntimePermission', 'oracle.DbmsJavaScriptUser', '' );

PL/SQL procedure successfully completed.

SQL> exec dbms_java.grant_permission( 'SRIRAM', 'SYS:java.lang.RuntimePermission', 'accessClassInPackage.jdk.nashorn.internal.runtime', '' );

PL/SQL procedure successfully completed.

SQL> exec dbms_java.grant_permission( 'SRIRAM', 'SYS:java.lang.reflect.ReflectPermission', 'suppressAccessChecks', '' );

PL/SQL procedure successfully completed.

SQL> select
  2
SQL> ed
Wrote file afiedt.buf

  1  CREATE or replace AND COMPILE JAVA SOURCE NAMED SQLFormatter AS
  2  import oracle.dbtools.app.Format;
  3  import java.sql.Connection;
  4  import java.sql.DriverManager;
  5  import java.sql.SQLException;
  6  import oracle.sql.BLOB;
  7  import oracle.sql.CLOB;
  8  import java.io.StringWriter;
  9  import java.io.PrintWriter;
 10  public class SQLFormatter {
 11      private static String getStackTrace(Exception e) {
 12         StringWriter writer = new StringWriter();
 13         PrintWriter printWriter = new PrintWriter( writer );
 14         e.printStackTrace( printWriter );
 15         printWriter.flush();
 16         return writer.toString();
 17      }
 18      public static Format getFormat() {
 19          oracle.dbtools.app.Format format = new oracle.dbtools.app.Format();
 20          format.options.put("singleLineComments", Format.InlineComments.CommentsUnchanged);
 21          format.options.put("kwCase", Format.Case.UPPER);
 22          format.options.put("idCase", Format.Case.NoCaseChange);
 23          format.options.put("adjustCaseOnly", false);
 24          format.options.put("formatThreshold", 1);
 25          format.options.put("alignTabColAliases", false);
 26          format.options.put("alignTypeDecl", true);
 27          format.options.put("alignNamedArgs", true);
 28          format.options.put("alignEquality", false);
 29          format.options.put("alignAssignments", true);
 30          format.options.put("alignRight", false);
 31          format.options.put("identSpaces", 3);
 32          format.options.put("useTab", false);
 33          format.options.put("breaksComma", Format.Breaks.Before);
 34          format.options.put("breaksProcArgs", false);
 35          format.options.put("breaksConcat", Format.Breaks.Before);
 36          format.options.put("breaksAroundLogicalConjunctions", Format.Breaks.Before);
 37          format.options.put("breaksAfterSelect", true);
 38          format.options.put("commasPerLine", 1);
 39          format.options.put("breakOnSubqueries", true);
 40          format.options.put("breakAnsiiJoin", true);
 41          format.options.put("breakParenCondition", true);
 42          format.options.put("maxCharLineSize", 120);
 43          format.options.put("forceLinebreaksBeforeComment", false);
 44          format.options.put("extraLinesAfterSignificantStatements", Format.BreaksX2.Keep);
 45          //format.options.put("flowControl", Format.FlowControl.IndentedActions);
 46          format.options.put("spaceAroundOperators", true);
 47          format.options.put("spaceAfterCommas", true);
 48          //format.options.put("spaceAroundBrackets", Format.Space.Default);
 49          return format;
 50      }
 51    public static String format(String str)
 52    {
 53      String res;
 54      try {
 55         Format f = SQLFormatter.getFormat();
 56         res = f.format(str);
 57         }
 58      catch (Exception e){
 59         res = "Error: " + e.getMessage() + " [ " + SQLFormatter.getStackTrace(e) + " ]";
 60      }
 61      return res;
 62    }
 63    public static CLOB formatClob(oracle.sql.CLOB clob)
 64    throws SQLException
 65    {
 66      String str = clob.getSubString(1, (int) clob.length());
 67      String res = SQLFormatter.format(str);
 68      Connection conn = DriverManager.getConnection("jdbc:default:connection:");
 69      CLOB resClob = CLOB.createTemporary(conn, false, BLOB.DURATION_SESSION);
 70      resClob.setString(1L, res);
 71      return resClob;
 72    }
 73* }
 74  /

Java created.

SQL> select
  2
SQL> ed
Wrote file afiedt.buf

  1  create or replace package SQLFormatter as
  2    FUNCTION Format(str in varchar2) RETURN VARCHAR2
  3    AS LANGUAGE JAVA NAME 'SQLFormatter.format(java.lang.String) return java.lang.String';
  4    FUNCTION FormatClob(str in clob) RETURN CLOB
  5    AS LANGUAGE JAVA NAME 'SQLFormatter.formatClob(oracle.sql.CLOB) return oracle.sql.CLOB';
  6* end;
  7  /

Package created.

SQL> desc SQLFormatter
FUNCTION FORMAT RETURNS VARCHAR2
 Argument Name                  Type                    In/Out Default?
 ------------------------------ ----------------------- ------ --------
 STR                            VARCHAR2                IN
FUNCTION FORMATCLOB RETURNS CLOB
 Argument Name                  Type                    In/Out Default?
 ------------------------------ ----------------------- ------ --------
 STR                            CLOB                    IN

SQL> select  sql_id||' - '||sql_text as "UnClear" from v$sqlarea where sql_id='gngtvs38t0060'
  2  union
  3  select sql_id||' - '||SQLFormatter.format(sql_text) "Readable" from v$sqlarea where sql_id='gngtvs38t0060'
  4  /

UnClear
------------------------------------------------------------------------------------------------------------------------
gngtvs38t0060 - SELECT /*+ CONNECT_BY_FILTERING */ s.privilege# FROM sys.sysauth$ s        CONNECT BY s.grantee# = PRIOR
 s.privilege#                                 AND (s.privilege# > 0 OR s.privilege# = -352)
 START WITH (s.privilege# > 0 OR s.privilege# = -352) AND s.grantee# IN     (SELECT c1.privilege# FROM sys.codeauth$ c1
WHERE c1.obj# = :1)            UNION                                                                      SELECT c2.priv
ilege# FROM sys.codeauth$ c2 WHERE c2.obj# = :2              ORDER BY 1 ASC









gngtvs38t0060 - SELECT /*+ CONNECT_BY_FILTERING */
   s.privilege#
FROM
   sys.sysauth$ s
CONNECT BY
   s.grantee# = PRIOR s.privilege#
AND (
         s.privilege# > 0
      OR s.privilege# =-352
   )
START WITH
   (
         s.privilege# > 0
      OR s.privilege# =-352
   )
AND s.grantee# IN (
      SELECT
         c1.privilege#
      FROM
         sys.codeauth$ c1
      WHERE
         c1.obj# =:1
   )
UNION
SELECT
   c2.privilege#
FROM
   sys.codeauth$ c2
WHERE
   c2.obj# =:2
ORDER BY 1 ASC


2 rows selected.

Refer http://orasql.org/2020/12/23/format-sql-or-pl-sql-directly-in-oracle-database/ for more info.

https://github.com/xtender/xt_scripts/tree/master/extra/SQLFormatter

Hope this is useful for you as well 🙂

Posted in formatter, sql | Tagged: formatter, java, oracle, plsql, sql | Leave a Comment »

Query for Top 10 by Version Count:

Posted by Sriram Sanka on September 13, 2022

set linesize 100
set pagesize 100
SELECT * FROM 
(SELECT substr(sql_text,1,40) sql,
        version_count, executions, hash_value,address
   FROM V$SQLAREA
  WHERE version_count > 20
 ORDER BY version_count DESC)
WHERE rownum  <=10
;

Posted in Uncategorized | Leave a Comment »

Query for Top 10 by Sharable Memory:

Posted by Sriram Sanka on September 13, 2022

set linesize 100
set pagesize 100
SELECT * FROM 
(SELECT substr(sql_text,1,40) sql,
        sharable_mem, executions, hash_value,address
   FROM V$SQLAREA
  WHERE sharable_mem > 1048576
 ORDER BY sharable_mem DESC)
WHERE rownum  <=10
;

Posted in Uncategorized | Leave a Comment »

« Previous Entries

Next Entries »

My Experiences with Databases & More

Oracle-MySQL-SQL SERVER-Python-Azure-AWS-Oracle Cloud-GCP etc

Moderator at

Member at

Subscribe

Categories

Archives

Visits n Views

Blogs I Follow

Follow Blog via Email

Total Views

$riram $anka

Fun with Python – Create Web Traffic using selenium & Python.

Hacking – FATDBA.COM ¯\_(ツ)_/¯ `ॐ`

Install MySQL 5.7 Community Server Using RPM Bundle- CENTOS 7

Upgrade MySQL Server from 5.7 to 8 CentOS 7 Linux

Quote from Mysql Website Upgrade Paths

Docker Part 1 – Install Docker CE and Docker Compose in Oracle Linux Server 8.6 – OCI-IAAS

Install & Configure SSL for Apache-Nginx using Lets Encrypt-CertBot.

Python Basics – Part 1

Language Introduction

String Split

List to String

SQL Formatter – Using Java/PL/SQL by Sayan Malakshinov

Query for Top 10 by Version Count:

Query for Top 10 by Sharable Memory:

Oracle-MySQL-SQL SERVER-Python-Azure-AWS-Oracle Cloud-GCP etc

Moderator at

Member at

Subscribe

Categories

Archives

Visits n Views

Blogs I Follow

Follow Blog via Email

Total Views

$riram $anka

Share this

Like this:

Share this

Like this:

Share this

Like this:

Quote from Mysql Website Upgrade Paths

Share this

Like this:

Share this

Like this:

Share this

Like this:

Language Introduction

String Split

List to String

Share this

Like this:

Share this

Like this:

Share this

Like this:

Share this

Like this: