How I can read conditions unserialised data in golang in map format?
[map[19:map[conditions:map[0:map[operator:== value:AMW-1900-50SLE-ROOM
is_value_processed:false type:feedexport/rule_condition_product
attribute:sku] 1:map[type:feedexport/rule_condition_product
attribute:sku operator:== value:ASL-B654-77-74-98-ROOM
is_value_processed:false] 2:map[is_value_processed:false
type:feedexport/rule_condition_product attribute:sku operator:==
value:ASL-B217-57-54S-95-ROOM]] type:feedexport/rule_condition_combine
attribute:<nil> operator:<nil> value:1 is_value_processed:<nil>
aggregator:any]]]
This is the code:
package main
import (
"fmt"
"github.com/wulijun/go-php-serialize/phpserialize"
)
func main() {
rules := RulesList()
for _, rule := range rules{
fmt.Println(rule.Conditions.(interface{}))
}
}
type Rule struct {
RuleId int `json:"rule_id"`
Conditions interface{} `json:"conditions"`
}
func RulesList() ([]Rule) {
db := DbConn()
res, err := db.Query(`SELECT r.rule_id, r.conditions_serialized AS
conditions FROM m_feedexport_rule AS r`)
CheckError(err)
rule := Rule{}
rules := []Rule{}
for res.Next() {
var ruleId int
var conditions string
err = res.Scan(&ruleId, &conditions)
CheckError(err)
cons, err := phpserialize.Decode(conditions)
CheckError(err)
rule.RuleId = ruleId
rule.Conditions = cons
rules = append(rules, rule)
}
return rules
}
The result is ok but I need it in map form, now this is the interface which I can't loop over. In case if anyone don't understand the code, ask me.
Thanks a lot.
You're talking about the type of the variable cons, are you?
Background
If yes, the reason its type is interface{} is because in PHP, it's possible to serialize a value of any type (from bare integer to a complicated object), and hence any deserialization code must cope with it. Since in Go the so-called "empty interface", interface{},
is satisfied by any type at all (including any custom type implemented by a programmer), it's sensbile for a decoder of PHP-serialized data to return a value of type interface{}.
Solution
After making sure decoding succeeded,
you need to either type-assert
the resulting value to a type you need or to use
a type switch
to diverge processing based on the concrete type of the
value returned by the decoder.
The approach is very well demonstrated by the
package you're using in its own test suite.
A snippet from it demonstrating the basic approach
if decodeRes, err = Decode(result); err != nil {
t.Errorf("decode data fail %v, %v", err, result)
return
}
decodeData, ok := decodeRes.(map[interface{}]interface{})
if !ok {
t.Errorf("decode data type error, expected: map[interface{}]interface{}, get: %T", decodeRes)
return
}
obj, _ := decodeData["object"].(*PhpObject)
if v, _ := obj.GetPrivateMemberValue("a"); v != int64(1) {
t.Errorf("object private value expected 1, get %v, %T", v, v)
}
if v := obj.GetClassName(); v != "A" {
t.Errorf("object class name expected A, get %v", v)
}
Here, decodeRes is what returned by the decoder.
That value is then type-asserted to make sure it's concrete type
(also called "dynamic" — meaning "at runtime") is
map[interface{}]interface{}.
Note that the so-called "two-argument" type assert is used
(it's also called a "comma-ok idiom") to not make the program
panic at runtime in case the concrete type is different from
the expected (always do this on data fetched from outside!).
The value of the type-asserted concrete type is stored in the
decodeData variable, and then that variable is inspected further.
Related
I've used file_get_contents for reading a WAV file in PHP and I want to use package github.com/mjibson/go-dsp/wav for same task in Go. But there is not any simple example about this package. I am new to Go and do not understand it. Is there anyone guiding me or suggest another way?
Code in PHP:
$wsdl = 'http://myApi.asmx?WSDL';
$client = new SoapClient($wsdl));
$data = file_get_contents(public_path() . "/forTest/record.wav");
$param = array(
'userName' => '***',
'password' => '***',
'file' => $data);
$res = $client->UploadMessage($param);
It looks like you do not need to use this package github.com/mjibson/go-dsp/wav. file_get_contents function is the preferred way to read the contents of a file into a string.
In Go, you can do something like this:
package main
import (
"fmt"
"io/ioutil"
)
func public_path() string {
return "/public/path/"
}
func main() {
dat, err := ioutil.ReadFile(public_path() + "/forTest/record.wav")
if err != nil {
fmt.Println(err)
}
fmt.Print(string(dat))
}
https://play.golang.org/p/l9R0940iK50
I think you just need to read the file, it really does not matter whether it is .wav or some other file. You can use go's builtin package io/ioutil.
Following is what you should do in go to read a disk file:
package main
import (
"fmt"
"io/ioutil"
"log"
)
func main() {
// Reading .wav file from disk.
fileData, err := ioutil.ReadFile("DISK_RELATIVE_PATH_PREFIX" + "/forTest/record.wav")
// ioutil.ReadFile returns two results,
// first one is data (slice of byte i.e. []byte) and the other one is error.
// If error is having nil value, you got successfully read the file,
// otherwise you need to handle the error.
if err != nil {
// Handle error here.
log.Fatal(err)
} else {
// Do whatever needed with the 'fileData' such as just print the data,
// or send it over the network.
fmt.Print(fileData)
}
}
Hope this helps.
Given a WAV file this returns its payload as a floating point audio curve along with essential audio file details like bit-depth, sample-rate and num-channels.
Had you simply read in the binary audio file using one of the underlying IO primitives in order to obtain the audio curve you would have to battle with your own bit shifting to pluck multi-byte ints as well as handling big or little endianness.
You still must be aware of how to handle multi-channel interleaving of audio samples from each channel which is not an issue for mono audio.
package main
import (
"fmt"
"os"
"github.com/youpy/go-wav"
"math"
)
func read_wav_file(input_file string, number_of_samples uint32) ([]float64, uint16, uint32, uint16) {
if number_of_samples == 0 {
number_of_samples = math.MaxInt32
}
blockAlign := 2
file, err := os.Open(input_file)
if err != nil {
panic(err)
}
reader := wav.NewReader(file)
wavformat, err_rd := reader.Format()
if err_rd != nil {
panic(err_rd)
}
if wavformat.AudioFormat != wav.AudioFormatPCM {
panic("Audio format is invalid ")
}
if int(wavformat.BlockAlign) != blockAlign {
fmt.Println("Block align is invalid ", wavformat.BlockAlign)
}
samples, err := reader.ReadSamples(number_of_samples) // must supply num samples w/o defaults to 2048
// // just supply a HUGE number then actual num is returned
wav_samples := make([]float64, 0)
for _, curr_sample := range samples {
wav_samples = append(wav_samples, reader.FloatValue(curr_sample, 0))
}
return wav_samples, wavformat.BitsPerSample, wavformat.SampleRate, wavformat.NumChannels
}
func main() {
input_audio := "/blah/genome_synth_evolved.wav"
audio_samples, bits_per_sample, input_audio_sample_rate, num_channels := read_wav_file( input_audio, 0)
fmt.Println("num samples ", len(audio_samples)/int(num_channels))
fmt.Println("bit depth ", bits_per_sample)
fmt.Println("sample rate ", input_audio_sample_rate)
fmt.Println("num channels", num_channels)
}
I've been having a bit of problems trying to figure out why my encryption is different in go compared to php and node. I was hoping someone could help me figure out the differences. Let's assume this is the data:
plaintext: hello big worldshello big worlds
key: jJr44P3WSM5F8AC573racFpzU5zj7Rg5
iv: 97iEhhtgVjoVwdUw
Here are the resulting encryptions in base64:
Node and PHP return :
OTdpRWhodGdWam9Wd2RVd0OgJ+Z7pSCVioYq41721jarxqLKXN3PcnnY6/AOrHeEfsTxXfCgm2uUi+vmCAdpvw==
Go returns:
OTdpRWhodGdWam9Wd2RVd0OgJ+Z7pSCVioYq41721jarxqLKXN3PcnnY6/AOrHeE
As you can see they're almost identical and its been driving me crazy. Could you guys take quick look at the encryption code below and give me hints on what the problem could be?
GO:
func EncryptString(plainstring string, keystring string, encFormat int, ivOverride bool) (string) {
// Load your secret key from a safe place and reuse it across multiple
// NewCipher calls. (Obviously don't use this example key for anything
// real.) If you want to convert a passphrase to a key, use a suitable
// package like bcrypt or scrypt.
key := []byte(keystring)
plaintext := []byte(plainstring)
// CBC mode works on blocks so plaintexts may need to be padded to the
// next whole block. For an example of such padding, see
// https://tools.ietf.org/html/rfc5246#section-6.2.3.2. Here we'll
// assume that the plaintext is already of the correct length.
if len(plaintext)%aes.BlockSize != 0 {
panic("plaintext is not a multiple of the block size")
}
block, err := aes.NewCipher(key)
if err != nil {
panic(err)
}
// The IV needs to be unique, but not secure. Therefore it's common to
// include it at the beginning of the ciphertext.
ciphertext := make([]byte, aes.BlockSize+len(plaintext))
iv := ciphertext[:aes.BlockSize]
if _, err := io.ReadFull(bytes.NewReader([]byte("97iEhhtgVjoVwdUw")), iv); err != nil {
panic(err)
}
mode := cipher.NewCBCEncrypter(block, iv)
mode.CryptBlocks(ciphertext[aes.BlockSize:], plaintext)
// It's important to remember that ciphertexts must be authenticated
// (i.e. by using crypto/hmac) as well as being encrypted in order to
// be secure.
return base64.StdEncoding.EncodeToString(ciphertext)
}
NODE:
encryptString: function(string, key, fmt = null, ivOverride = false) {
// Build an initialisation vector
let iv;
if(!ivOverride) {
iv = crypto.randomBytes(IV_NUM_BYTES).toString('hex').slice(0,16);
} else {
iv = IV_OVERRIDE_VALUE; //97iEhhtgVjoVwdUw
}
// and encrypt
let encryptor = crypto.createCipheriv('aes-256-cbc', key, iv);
let encryptedData = encryptor.update(string, 'utf8', 'binary') + encryptor.final('binary');
encryptedData = iv+''+encryptedData;
encryptedData = Buffer.from(encryptedData, 'binary').toString('base64');
return encryptedData;
}
I've noticed that removing encryptor.final('binary') makes the two result in the same encryption but php does not have the .final() thing going for it. Php uses open_ssl_encrypt() which seems to have this built in. Is there a way to add an equivalent in go? Looking for advice. Thanks
Okay, I managed to get the go output to match your other outputs, but I'm not 100% clear on all of the details--in particular why the PHP and Node versions behave the way they do (your output from the Go version seems like the "correct" result in my mind, as I'll explain).
My first observation was that your output from Node and PHP was longer than the Go version, by approximately one block length, and only the tail end is different. This told me that, somehow, those versions are being padded more than the go version.
So, I tried padding the Go version according to the default padding scheme used by PHP and Node, PKCS#7. Basically, if you need to pad by 5 bytes, then each of the padding bytes should be equal to 0x05, 6 bytes are padded with 0x06, etc. Go's default aes.BlockSize is equal to 16, so I tried padding your input string with 16 0x10 bytes. This led to the correct answer!
Honestly, not padding the input at all if it's already block-aligned is understandable behavior, but apparently Node and PHP follow RFC 5652 and always add padding (see the edit), even if they need to add another entire block just of padding.
Here's the Go code to make your outputs match:
package main
import (
"bytes"
"crypto/aes"
"crypto/cipher"
"encoding/base64"
"fmt"
"io"
)
// Based on Wikipedia: https://en.wikipedia.org/wiki/Padding_(cryptography)#PKCS7
func PadToBlockSize(input string) string {
paddingNeeded := aes.BlockSize - (len(input) % aes.BlockSize)
if paddingNeeded >= 256 {
panic("I'm too lazy to handle this case for the sake of an example :)")
}
if paddingNeeded == 0 {
paddingNeeded = aes.BlockSize
}
// Inefficient, once again, this is an example only!
for i := 0; i < paddingNeeded; i++ {
input += string(byte(paddingNeeded))
}
return input
}
// (Identical to your code, I just deleted comments to save space)
func EncryptString(plainstring string, keystring string, encFormat int, ivOverride bool) string {
key := []byte(keystring)
plaintext := []byte(plainstring)
if len(plaintext)%aes.BlockSize != 0 {
panic("plaintext is not a multiple of the block size")
}
block, err := aes.NewCipher(key)
if err != nil {
panic(err)
}
ciphertext := make([]byte, aes.BlockSize+len(plaintext))
iv := ciphertext[:aes.BlockSize]
if _, err := io.ReadFull(bytes.NewReader([]byte("97iEhhtgVjoVwdUw")), iv); err != nil {
panic(err)
}
mode := cipher.NewCBCEncrypter(block, iv)
mode.CryptBlocks(ciphertext[aes.BlockSize:], plaintext)
return base64.StdEncoding.EncodeToString(ciphertext)
}
func main() {
plaintext := "hello big worldshello big worlds"
key := "jJr44P3WSM5F8AC573racFpzU5zj7Rg5"
phpText := "OTdpRWhodGdWam9Wd2RVd0OgJ+Z7pSCVioYq41721jarxqLKXN3PcnnY6/AOrHeEfsTxXfCgm2uUi+vmCAdpvw=="
fmt.Println("Go : " + EncryptString(PadToBlockSize(plaintext), key, 0, false))
fmt.Println("PHP: " + phpText)
}
Edit:
Actually, it looks like Node and PHP are just following RFC 5652 correctly, which mandates that all input needs to be padded. The fact that padding will always be present, even if the input was block-aligned, disambiguates decryption. Go just leaves the padding step to the user.
I need to get values from a serialized string which generated from php code
So I use a package named:php_serialize to unserialize the string and then got a result of interface{} type .
But I have no idea how to get values inside the result.
This is code:
package main
import (
"github.com/yvasiyarov/php_session_decoder/php_serialize"
"fmt"
)
func main() {
// this string is generated from php code
str := `a:3:{s:4:"name";s:3:"tom";s:3:"age";s:2:"23";s:7:"friends";a:2:{i:0;a:1:{s:4:"name";s:5:"jerry";}i:1;a:1:{s:4:"name";s:4:"jack";}}}`
decoder := php_serialize.NewUnSerializer(str)
if result, err := decoder.Decode(); err != nil {
panic(err)
} else {
fmt.Println(result)
}
}
The print result is :
map[name:tom age:23 friends:map[0:map[name:jerry] 1:map[name:jack]]]
This result is a php_serialize.PhpValue type, which is interface{} type
The next step is how to get values inside the result?
such as get the age field and value
You must assert the result to map[string]interface:
mResult := result.(map[string]interface{})
fmt.Println(mResult["name"])
And once more assertion for friends:
mFriends := mResult["friends"].(map[int]map[string]interface{})
Then use it: mFriends[0]["name"]
Here some ways to access the data:
package main
import (
"fmt"
"github.com/yvasiyarov/php_session_decoder/php_serialize"
)
func main() {
// this string is generated from php code
str := `a:3:{s:4:"name";s:3:"tom";s:3:"age";s:2:"23";s:7:"friends";a:2:{i:0;a:1:{s:4:"name";s:5:"jerry";}i:1;a:1:{s:4:"name";s:4:"jack";}}}`
decoder := php_serialize.NewUnSerializer(str)
result, err := decoder.Decode()
if err != nil {
panic(err)
}
fmt.Println(result)
// simple assert
t := result.(php_serialize.PhpArray)
// use php_seriale build in function to get string
strVal := php_serialize.PhpValueString(t["name"])
fmt.Println(strVal)
// type switch in case of different valid types
switch t := result.(type) {
default:
fmt.Printf("unexpected type %T\n", t) // %T prints whatever type t has
case php_serialize.PhpArray:
fmt.Println(t)
fmt.Println(t["name"])
fmt.Println(t["age"])
// should be done recursively...
switch f := t["friends"].(type) {
default:
fmt.Printf("unexpected type %T\n", f) // %T prints whatever type t has
case php_serialize.PhpArray:
fmt.Println(f)
fmt.Println(f[0])
fmt.Println(f[1])
}
}
}
I hope this gives you some ideas.
Basic concept
php_serialize has built in functions to convert primitives.
Variable structures are represented with built in types which need to be used to access the structure.
I was hoping someone had already implemented this in golang as I am far from even good at cryptography. However in porting a project from php to golang I have run into an issue with porting the openssl_encrypt method found here. I have also dug into the source code a little with no avail.
Here is the method I have implemented in golang. which gives me the output
lvb7JwaI4OCYUrdJMm8Q9uDd9rIILnvbZKJb/ozFbwCmLKkxoJN5Zf/ODOJ/RGq5
Here is the output I need when using php.
lvb7JwaI4OCYUrdJMm8Q9uDd9rIILnvbZKJb/ozFbwDV98XaJjvzEjBQp7jc+2DH
And here is the function I used to generate it with php.
$data = "This is some text I want to encrypt";
$method = "aes-256-cbc";
$password = "This is a really long key and su";
$options = 0;
$iv = "MMMMMMMMMMMMMMMM";
echo openssl_encrypt($data, $method, $password, $options, $iv);
To me it looks like it is very close and I must be missing something obvious.
You were very close, but you had the padding wrong. According to this answer (and the PHP docs), PHP uses the default OpenSSL padding behavior, which is to use the required number of padding bytes as the padding byte value.
The only change I made was:
copy(plaintextblock[length:], bytes.Repeat([]byte{uint8(extendBlock)}, extendBlock))
You can see the full updated code here.
Others beat me to the answer while I was playing with it, but I have a "better" fixed version of your example code that also takes into account that padding is always required (at least to emulate what the php code does).
It also shows the openssl command line that you'd use to do the same thing, and if available runs it (of course the playground won't).
package main
import (
"crypto/aes"
"crypto/cipher"
"encoding/base64"
"fmt"
"log"
"os/exec"
"strings"
)
func main() {
const input = "This is some text I want to encrypt"
fmt.Println(opensslCommand(input))
fmt.Println(aesCBCenctypt(input))
}
func aesCBCenctypt(input string) string {
// Of course real IVs should be from crypto/rand
iv := []byte("MMMMMMMMMMMMMMMM")
// And real keys should be from something like PBKDF2, RFC 2898.
// E.g. use golang.org/x/crypto/pbkdf2 to turn a
// "passphrase" into a key.
key := []byte("This is a really long key and su")
// Make sure the block size is a multiple of aes.BlockSize
// Pad to aes.BlockSize using the pad length as the padding
// byte. If we would otherwise need no padding we instead
// pad an entire extra block.
pad := (aes.BlockSize - len(input)%aes.BlockSize)
if pad == 0 {
pad = aes.BlockSize
}
data := make([]byte, len(input)+pad)
copy(data, input)
for i := len(input); i < len(input)+pad; i++ {
data[i] = byte(pad)
}
cb, err := aes.NewCipher(key)
if err != nil {
log.Fatalln("error NewCipher():", err)
}
mode := cipher.NewCBCEncrypter(cb, iv)
mode.CryptBlocks(data, data)
return base64.StdEncoding.EncodeToString(data)
}
// Just for comparison, don't do this for real!
func opensslCommand(input string) string {
iv := []byte("MMMMMMMMMMMMMMMM")
key := []byte("This is a really long key and su")
args := []string{"enc", "-aes-256-cbc", "-base64"}
// "-nosalt", "-nopad"
args = append(args, "-iv", fmt.Sprintf("%X", iv))
args = append(args, "-K", fmt.Sprintf("%X", key))
cmd := exec.Command("openssl", args...)
// Show how you could do this via the command line:
fmt.Println("Command:", strings.Join(cmd.Args, " "))
cmd.Stdin = strings.NewReader(input)
result, err := cmd.CombinedOutput()
if err != nil {
if e, ok := err.(*exec.Error); ok && e.Err == exec.ErrNotFound {
// openssl not available
return err.Error() // XXX
}
// some other error, show it and the (error?) output and die
fmt.Println("cmd error:", err)
log.Fatalf("result %q", result)
}
// Strip trailing '\n' and return it.
if n := len(result) - 1; result[n] == '\n' {
result = result[:n]
}
return string(result)
}
Playground
Note: I am limited to PHP <-> VBA. Please do not suggest anything that requires an Excel Addon, or any other language/method.
I have a function that connect to a specified URL, submits data, and then retrieves other data. This works great. I'm trying to write it so i can use it as a generic function I can use to connect to any file I need to connect to - each would return different data (one could be user data, one could be complex calculations etc).
When it retrieves the data from PHP, is there a way to dynamically set the variables based on what is received - even if i do not know what has been received.
I can make PHP return to VBA the string in any format, so I'm using the below as an example:
String that is received in vba:
myValue1=Dave&someOtherValue=Hockey&HockeyDate=Yesterday
If i were to parse this in PHP, I could do something similar to (not accurate, just written for example purposes);
$myData = "myValue1=Dave&someOtherValue=Hockey&HockeyDate=Yesterday"
$myArr = explode("&",$myData)
foreach($myArr as $key => $value){
${$key} = $value;
}
echo $someOtherValue; //Would output to the screen 'Hockey';
I would like to do something similar in VBA. The string I am receiving is from a PHP file, so I can format it any way (json etc etc), I just essentially want to be able to define the VARIABLES when outputting the string from PHP. Is this possible in VBA?.
The current state of the function I have that is working great for connections is as below:-
Function kick_connect(url As String, formdata)
'On Error GoTo connectError
Dim http
Set http = CreateObject("MSXML2.XMLHTTP")
http.Open "POST", url, False
http.setRequestHeader "Content-Type", "application/x-www-form-urlencoded"
http.send (formdata)
kick_connect = http.responseText
Exit Function
connectError:
kick_connect = False
End Function
Ultimately, I want to be able to do something like
sub mySub
myData = "getId=" & Range("A1").Value
myValue = kick_connect("http://path-to-my-php-file.php",myData)
if myValue = False then
'Handle connection error here
exit sub
end if
'do something snazzy here to split "myValue" string (eg "myValue1=Dave&someOtherValue=Hockey&HockeyDate=Yesterday") into own variables
msgbox(myValue1) 'Should output "Dave"
end sub
Obviously I could put the values into an array, and reference that, however I specifically want to know if this exact thing is possible, to allow for flexibility with the scripts that already exist.
I hope this makes sense, and am really grateful for any replies i get.
Thank you.
You can use a Collection:
Dim Tmp As String
Dim s As String
Dim i As Integer
Dim colVariabili As New Collection
Tmp = "myValue1=Dave&someOtherValue=Hockey&HockeyDate=Yesterday"
Dim FieldStr() As String
Dim FieldSplitStr() As String
FieldStr = Split(Tmp, "&")
For Each xx In FieldStr
FieldSplitStr = Split(xx, "=")
colVariabili.Add FieldSplitStr(1), FieldSplitStr(0)
Next
Debug.Print colVariabili("myValue1")
Debug.Print colVariabili("someOtherValue")
Debug.Print colVariabili("HockeyDate")
It's ok if you don't have the correct sequence of var...
I am not sure if this can help you, but as far as I understand your question you want to be able to create the variables dynamically based on the query string parameters. If so then here is example how to add this variables dynamically. Code needs standard module with a name 'QueryStringVariables'. In this module the query string will be parsed and each query string parameter will be added as get-property. If you wish to be able to change the value as well then you will need to add let-property as well.
Add reference to Microsoft Visual Basic For Applications Extensibility
Option Explicit
Private Const SourceQueryString As String = "myValue1=Dave&someOtherValue=Hockey&HockeyDate=Yesterday"
Sub Test()
Dim queryStringVariablesComponent As VBIDE.vbComponent
Dim queryStringVariablesModule As VBIDE.CodeModule
Dim codeText As String
Dim lineNum As Long: lineNum = 1
Dim lineCount As Long
Set queryStringVariablesComponent = ThisWorkbook.VBProject.VBComponents("QueryStringVariables")
Set queryStringVariablesModule = queryStringVariablesComponent.CodeModule
queryStringVariablesModule.DeleteLines 1, queryStringVariablesModule.CountOfLines
Dim parts
parts = Split(SourceQueryString, "&")
Dim part, variableName, variableValue
For Each part In parts
variableName = Split(part, "=")(0)
variableValue = Split(part, "=")(1)
codeText = "Public Property Get " & variableName & "() As String"
queryStringVariablesModule.InsertLines lineNum, codeText
lineNum = lineNum + 1
codeText = variableName & " = """ & variableValue & ""
queryStringVariablesModule.InsertLines lineNum, codeText
lineNum = lineNum + 1
codeText = "End Property"
queryStringVariablesModule.InsertLines lineNum, codeText
lineNum = lineNum + 1
Next
DisplayIt
End Sub
Sub DisplayIt()
MsgBox myValue1 'Should output "Dave"
End Sub